2026-05-11T15:51:10,150 Created temporary directory: /tmp/pip-ephem-wheel-cache-fdlbw67q 2026-05-11T15:51:10,152 Created temporary directory: /tmp/pip-build-tracker-gray8eve 2026-05-11T15:51:10,152 Initialized build tracking at /tmp/pip-build-tracker-gray8eve 2026-05-11T15:51:10,153 Created build tracker: /tmp/pip-build-tracker-gray8eve 2026-05-11T15:51:10,153 Entered build tracker: /tmp/pip-build-tracker-gray8eve 2026-05-11T15:51:10,154 Created temporary directory: /tmp/pip-wheel-3__ovqck 2026-05-11T15:51:10,157 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2026-05-11T15:51:10,160 Created temporary directory: /tmp/pip-ephem-wheel-cache-drshsi6r 2026-05-11T15:51:10,182 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2026-05-11T15:51:10,185 2 location(s) to search for versions of lm-eval: 2026-05-11T15:51:10,185 * https://pypi.org/simple/lm-eval/ 2026-05-11T15:51:10,185 * https://www.piwheels.org/simple/lm-eval/ 2026-05-11T15:51:10,186 Fetching project page and analyzing links: https://pypi.org/simple/lm-eval/ 2026-05-11T15:51:10,187 Getting page https://pypi.org/simple/lm-eval/ 2026-05-11T15:51:10,189 Found index url https://pypi.org/simple 2026-05-11T15:51:10,337 Fetched page https://pypi.org/simple/lm-eval/ as application/vnd.pypi.simple.v1+json 2026-05-11T15:51:10,344 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/1b/5f/7841febb99c12ffb453d33a67b9841e89dba18c388b644bf22b81d137fc4/lm_eval-0.0.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T15:51:10,345 Found link https://files.pythonhosted.org/packages/21/5a/feb5ff3a1591ca963c54873d39116b0e6a4f80e493e961ac08569709c5d7/lm_eval-0.0.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.0.1 2026-05-11T15:51:10,346 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/f3/a7/63cbce8b51de25fabb1c49f3a3fd1704faaacadb5ed816401f800e4d2dbd/lm_eval-0.2.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T15:51:10,347 Found link https://files.pythonhosted.org/packages/c5/fd/edd21b0f258b4ec0260f99f5b2ac3864f7cddc8fb7c83bbb2379a6aab975/lm_eval-0.2.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.2.0 2026-05-11T15:51:10,348 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/61/c5/bff92e6b61fc2b0c1b7ac769633731910152e5176a404912ce7c07329ba0/lm_eval-0.3.0-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T15:51:10,349 Found link https://files.pythonhosted.org/packages/c4/f8/58abc65390a758c8c2e5f1d8bb9b58d7885d02535d5f48de27006453d07e/lm_eval-0.3.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.6), version: 0.3.0 2026-05-11T15:51:10,350 Found link https://files.pythonhosted.org/packages/45/e0/05001c2e56e2f8f793189442176432ddb89a2f1f1ef00ea154d0bc00fe37/lm_eval-0.4.0.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.0 2026-05-11T15:51:10,351 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/49/2d/39f7a25ab663cb45cfc7773b85980f01df44853cc427d00dce94c90b43e6/lm_eval-0.4.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,352 Found link https://files.pythonhosted.org/packages/5a/02/1c7f1ac2f139f4c05af5b94e2c4f88a70404fa0b0c22a5fb04dec0216b03/lm_eval-0.4.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.1 2026-05-11T15:51:10,352 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/0c/5a/64cf703b62ac7ada09a514c16c7136bb4ea7ef3030eb4c3d780900a5a634/lm_eval-0.4.2-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,353 Found link https://files.pythonhosted.org/packages/75/ca/18814743ba3b42d19d8524a4f771e9c3a7aa02cd4c579747a0f513907205/lm_eval-0.4.2.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.2 2026-05-11T15:51:10,354 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/39/2b/eb15fb57ba48b3cebfd302713d680d9917101867e2b5fbfd266e88b89ff3/lm_eval-0.4.3-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,355 Found link https://files.pythonhosted.org/packages/2e/69/dbff28f17e17521735460b25645479e85b58d8ad7ea8488e4672ab52b58b/lm_eval-0.4.3.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.3 2026-05-11T15:51:10,355 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/ad/e1/ac8d7d26b4d6a9eea8859d438691b5d8209a37be2117b3e61f247de2d9e7/lm_eval-0.4.4-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,356 Found link https://files.pythonhosted.org/packages/c6/a8/4dcffd9ad0ddb6c479f93d8316a108a6f9b4d81756c892f150b6c307e8d4/lm_eval-0.4.4.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.4 2026-05-11T15:51:10,357 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/c1/78/fb3ea82c77ccc4dbbd6cafceb15ca01c1cd4d778217d572d65c919b0643f/lm_eval-0.4.5-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,358 Found link https://files.pythonhosted.org/packages/68/7a/cb1993a8d217a5b4c6e52f612a677c4f02c3412d2e3093ecbf6f6413136c/lm_eval-0.4.5.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.5 2026-05-11T15:51:10,358 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/45/b9/1b4e3268b590d9ff16e087685d9526455bb677c3e4d0caeba4451f20c586/lm_eval-0.4.7-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,359 Found link https://files.pythonhosted.org/packages/53/c9/b5d03d5b2bf6819008e377844999fbd04ab00dff0c43728957f1c90a53c5/lm_eval-0.4.7.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.8), version: 0.4.7 2026-05-11T15:51:10,360 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/c3/0b/36d6117f644f3685e6b87005ecd7051d01e9cdcf617e8e671102c1546de2/lm_eval-0.4.8-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T15:51:10,361 Found link https://files.pythonhosted.org/packages/ba/b2/fe23d318c5f645636f9f776c373d1ba68aa55c998894660613a09b0fe2ec/lm_eval-0.4.8.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9), version: 0.4.8 2026-05-11T15:51:10,362 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/b6/4f/3ab58de3dfee26f693d149824d9d62723f771392c4756becccbcbaaa4e03/lm_eval-0.4.9-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T15:51:10,363 Found link https://files.pythonhosted.org/packages/e5/ab/866d8957cddbb26510e5cf5f7c5ec01e179f82d54d2dafd3a6c5c31fc200/lm_eval-0.4.9.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9), version: 0.4.9 2026-05-11T15:51:10,364 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/18/ba/3a992d435deaf5461f50417bd7ba38a21831a04520941a9ecf1020dcf900/lm_eval-0.4.9.1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T15:51:10,364 Found link https://files.pythonhosted.org/packages/b9/81/943efd4d008080625ddb135d961fc40e78fa75c61f4ce1b4b1244c667571/lm_eval-0.4.9.1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.9), version: 0.4.9.1 2026-05-11T15:51:10,365 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/6c/50/3903694e69c94ae8dcec654843cb99799e45137e0d1a0a581009aa592673/lm_eval-0.4.9.2-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,366 Found link https://files.pythonhosted.org/packages/e1/5e/e6d70f4aa2acd7932a873c51de93ddf2dbd1c43aa6f6c86314ab3c0d279d/lm_eval-0.4.9.2.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.9.2 2026-05-11T15:51:10,367 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/23/5b/d0762a01f140e945aa7a44c92bb0c5c149c2c2917278c02eef6f5053b8e1/lm_eval-0.4.10-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,368 Found link https://files.pythonhosted.org/packages/7c/1c/858cc03ac7960bc7a1323d40a0d912ebef560825075504238f6721ad8a3a/lm_eval-0.4.10.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.10 2026-05-11T15:51:10,368 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/df/b3/8c2923ad4c4d911307e20ab9c7d7869d3811b5a76fe6dbe53678aafbeb04/lm_eval-0.4.11-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,369 Found link https://files.pythonhosted.org/packages/18/40/ad60ae17f97902ba4ee67914a94c159552f8643fb4598d1d2430a571f0a2/lm_eval-0.4.11.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.11 2026-05-11T15:51:10,370 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/bd/43/944724465ed73ec2889f58007c114ce8dd2023324bc86d3e6f41ce00e2e8/lm_eval-0.4.12-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,370 Found link https://files.pythonhosted.org/packages/12/32/783623b1d1fe7c89ebd774db46bef4c787fe93e7c5e1b4877c7dfbf1abbe/lm_eval-0.4.12.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.4.12 2026-05-11T15:51:10,371 Skipping link: No binaries permitted for lm-eval: https://files.pythonhosted.org/packages/f4/dc/8dc6e399a1e97948064a571639ad5b3f2b7cae84f930dae081e49bcafd47/lm_eval-0.5.0.dev1-py3-none-any.whl (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,372 Found link https://files.pythonhosted.org/packages/65/41/26141271560b0a41e2279fe2a2f29bf75e26cfb93849e3d5ad68c28ba66b/lm_eval-0.5.0.dev1.tar.gz (from https://pypi.org/simple/lm-eval/) (requires-python:>=3.10), version: 0.5.0.dev1 2026-05-11T15:51:10,373 Fetching project page and analyzing links: https://www.piwheels.org/simple/lm-eval/ 2026-05-11T15:51:10,374 Getting page https://www.piwheels.org/simple/lm-eval/ 2026-05-11T15:51:10,375 Found index url https://www.piwheels.org/simple 2026-05-11T15:51:10,546 Fetched page https://www.piwheels.org/simple/lm-eval/ as text/html 2026-05-11T15:51:10,552 Skipping link: No binaries permitted for lm-eval: https://www.piwheels.org/simple/lm-eval/lm_eval-0.4.12-py3-none-any.whl#sha256=fcb805fe553443c0365cbb87826c85649f35c9f7d40fb07e0272f8629b90cb01 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,553 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.11-py3-none-any.whl#sha256=7a994cec7f884dff2892c895be7943f2e5daf899d1a0c96862a09e86d38317ef (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,553 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.10-py3-none-any.whl#sha256=f974c8439950d7e007710fae43c1ff085ba2724b8c9c29456987c442a14ddc4a (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,554 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.9.2-py3-none-any.whl#sha256=6433acae848f5040a05aec1f8b4d946c1874207195592064a0dccbf11e6a3bf7 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.10) 2026-05-11T15:51:10,555 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.9.1-py3-none-any.whl#sha256=97ecff896cf8c8a3461730701532cb297638f594631d30e0a0e64bedacc771ce (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T15:51:10,556 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.9-py3-none-any.whl#sha256=aae7521fd8819d6dd3ae674c07816f449a70669bef1db6cdce2968c21ad1f7af (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T15:51:10,556 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.8-py3-none-any.whl#sha256=1ed005366d8f0154fa200db6e0eb56cc5ef33df2bbd2705d19c742e4283593c8 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.9) 2026-05-11T15:51:10,556 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.7-py3-none-any.whl#sha256=3b462a73021c5411e6b4491d07d5c4c009843435a2c9350ae5949fbf4a196ed9 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,557 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.5-py3-none-any.whl#sha256=c1292031153f79a22db4193ed91074a1c57ffaec3d9768215a455efd53ff5f02 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,557 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.4-py3-none-any.whl#sha256=1a6e6b514b470ad4c795439aa4cf9cb88b1360d915c7b3108ae5b47e3f7b36fc (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,558 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.3-py3-none-any.whl#sha256=7f4cb7cbd7ea20f2624f1315e69dfbc884cc187749c85760f9b4ec03e7efe491 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,558 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.2-py3-none-any.whl#sha256=a0bcaff492a09947d8a2b80400f224113b1e313da6a688084711130da18e1af7 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,559 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.1-py3-none-any.whl#sha256=796615edf52f5673187a884a9d77c7d755fe95ca2d16fe77a1f042730eb957b3 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,560 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.4.0-py3-none-any.whl#sha256=1a863b5f478f2e66e921dbbf2e4dcda06a923021763f01b59c5b03bdd3af01c2 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.8) 2026-05-11T15:51:10,560 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.3.0-py3-none-any.whl#sha256=498b8b8954c1f9c17f46e3ec096e9be6b9c96ee70560ee613a4eb9c7b9d31644 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T15:51:10,561 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.2.0-py3-none-any.whl#sha256=e06d3d7b6016be832e6889cbc9c4787b99156ba57f8feb31d3aeb27304c6558c (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T15:51:10,561 Skipping link: No binaries permitted for lm-eval: https://archive1.piwheels.org/simple/lm-eval/lm_eval-0.0.1-py3-none-any.whl#sha256=0afc289f69286f71017fb9811dfea6cda7c703bf693ad43106fbbff1f164cf14 (from https://www.piwheels.org/simple/lm-eval/) (requires-python:>=3.6) 2026-05-11T15:51:10,562 Skipping link: not a file: https://www.piwheels.org/simple/lm-eval/ 2026-05-11T15:51:10,563 Skipping link: not a file: https://pypi.org/simple/lm-eval/ 2026-05-11T15:51:10,585 Given no hashes to check 1 links for project 'lm-eval': discarding no candidates 2026-05-11T15:51:10,605 Collecting lm-eval==0.5.0.dev1 2026-05-11T15:51:10,608 Created temporary directory: /tmp/pip-unpack-aqyeg5hj 2026-05-11T15:51:10,836 Downloading lm_eval-0.5.0.dev1.tar.gz (3.4 MB) 2026-05-11T15:51:35,042 Added lm-eval==0.5.0.dev1 from https://files.pythonhosted.org/packages/65/41/26141271560b0a41e2279fe2a2f29bf75e26cfb93849e3d5ad68c28ba66b/lm_eval-0.5.0.dev1.tar.gz to build tracker '/tmp/pip-build-tracker-gray8eve' 2026-05-11T15:51:35,050 Created temporary directory: /tmp/pip-build-env-jpy2cbbz 2026-05-11T15:51:35,055 Installing build dependencies: started 2026-05-11T15:51:35,056 Running command pip subprocess to install build dependencies 2026-05-11T15:51:36,374 Using pip 23.0.1 from /usr/lib/python3/dist-packages/pip (python 3.11) 2026-05-11T15:51:36,876 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2026-05-11T15:51:36,903 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2026-05-11T15:51:38,640 Collecting setuptools>=82.0.0 2026-05-11T15:51:38,716 Using cached https://www.piwheels.org/simple/setuptools/setuptools-82.0.1-py3-none-any.whl (1.0 MB) 2026-05-11T15:51:41,699 Installing collected packages: setuptools 2026-05-11T15:51:44,853 Successfully installed setuptools-82.0.1 2026-05-11T15:51:45,119 Installing build dependencies: finished with status 'done' 2026-05-11T15:51:45,126 Getting requirements to build wheel: started 2026-05-11T15:51:45,127 Running command Getting requirements to build wheel 2026-05-11T15:51:46,436 /tmp/pip-build-env-jpy2cbbz/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2026-05-11T15:51:46,437 !! 2026-05-11T15:51:46,438 ******************************************************************************** 2026-05-11T15:51:46,439 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2026-05-11T15:51:46,440 By 2027-Feb-18, you need to update your project and remove deprecated calls 2026-05-11T15:51:46,440 or your builds will no longer be supported. 2026-05-11T15:51:46,442 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2026-05-11T15:51:46,442 ******************************************************************************** 2026-05-11T15:51:46,443 !! 2026-05-11T15:51:46,444 corresp(dist, value, root_dir) 2026-05-11T15:51:46,531 running egg_info 2026-05-11T15:51:46,537 writing lm_eval.egg-info/PKG-INFO 2026-05-11T15:51:46,559 writing dependency_links to lm_eval.egg-info/dependency_links.txt 2026-05-11T15:51:46,561 writing entry points to lm_eval.egg-info/entry_points.txt 2026-05-11T15:51:46,571 writing requirements to lm_eval.egg-info/requires.txt 2026-05-11T15:51:46,573 writing top-level names to lm_eval.egg-info/top_level.txt 2026-05-11T15:51:49,798 reading manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:51:50,529 reading manifest template 'MANIFEST.in' 2026-05-11T15:51:50,530 warning: manifest_maker: MANIFEST.in, line 1: 'recursive-include' expects ... 2026-05-11T15:51:50,532 adding license file 'LICENSE.md' 2026-05-11T15:51:51,332 writing manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:51:51,471 Getting requirements to build wheel: finished with status 'done' 2026-05-11T15:51:51,475 Created temporary directory: /tmp/pip-modern-metadata-npqusfb6 2026-05-11T15:51:51,477 Preparing metadata (pyproject.toml): started 2026-05-11T15:51:51,478 Running command Preparing metadata (pyproject.toml) 2026-05-11T15:51:52,328 /tmp/pip-build-env-jpy2cbbz/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2026-05-11T15:51:52,329 !! 2026-05-11T15:51:52,330 ******************************************************************************** 2026-05-11T15:51:52,330 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2026-05-11T15:51:52,331 By 2027-Feb-18, you need to update your project and remove deprecated calls 2026-05-11T15:51:52,331 or your builds will no longer be supported. 2026-05-11T15:51:52,332 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2026-05-11T15:51:52,332 ******************************************************************************** 2026-05-11T15:51:52,333 !! 2026-05-11T15:51:52,334 corresp(dist, value, root_dir) 2026-05-11T15:51:52,426 running dist_info 2026-05-11T15:51:52,436 creating /tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info 2026-05-11T15:51:52,437 writing /tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/PKG-INFO 2026-05-11T15:51:52,458 writing dependency_links to /tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/dependency_links.txt 2026-05-11T15:51:52,460 writing entry points to /tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/entry_points.txt 2026-05-11T15:51:52,470 writing requirements to /tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/requires.txt 2026-05-11T15:51:52,472 writing top-level names to /tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/top_level.txt 2026-05-11T15:51:52,473 writing manifest file '/tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:51:57,904 reading manifest file '/tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:51:57,906 reading manifest template 'MANIFEST.in' 2026-05-11T15:51:57,907 warning: manifest_maker: MANIFEST.in, line 1: 'recursive-include' expects ... 2026-05-11T15:51:57,908 adding license file 'LICENSE.md' 2026-05-11T15:51:58,410 writing manifest file '/tmp/pip-modern-metadata-npqusfb6/lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:51:58,435 creating '/tmp/pip-modern-metadata-npqusfb6/lm_eval-0.5.0.dev1.dist-info' 2026-05-11T15:51:58,593 Preparing metadata (pyproject.toml): finished with status 'done' 2026-05-11T15:51:58,602 Source in /tmp/pip-wheel-3__ovqck/lm-eval_2358f874be214a168ceae0c6f928a752 has version 0.5.0.dev1, which satisfies requirement lm-eval==0.5.0.dev1 from https://files.pythonhosted.org/packages/65/41/26141271560b0a41e2279fe2a2f29bf75e26cfb93849e3d5ad68c28ba66b/lm_eval-0.5.0.dev1.tar.gz 2026-05-11T15:51:58,604 Removed lm-eval==0.5.0.dev1 from https://files.pythonhosted.org/packages/65/41/26141271560b0a41e2279fe2a2f29bf75e26cfb93849e3d5ad68c28ba66b/lm_eval-0.5.0.dev1.tar.gz from build tracker '/tmp/pip-build-tracker-gray8eve' 2026-05-11T15:51:58,617 Created temporary directory: /tmp/pip-unpack-dhb2y8v5 2026-05-11T15:51:58,617 Building wheels for collected packages: lm-eval 2026-05-11T15:51:58,622 Created temporary directory: /tmp/pip-wheel-0biwgdc6 2026-05-11T15:51:58,622 Destination directory: /tmp/pip-wheel-0biwgdc6 2026-05-11T15:51:58,625 Building wheel for lm-eval (pyproject.toml): started 2026-05-11T15:51:58,626 Running command Building wheel for lm-eval (pyproject.toml) 2026-05-11T15:51:59,445 /tmp/pip-build-env-jpy2cbbz/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2026-05-11T15:51:59,445 !! 2026-05-11T15:51:59,446 ******************************************************************************** 2026-05-11T15:51:59,447 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2026-05-11T15:51:59,449 By 2027-Feb-18, you need to update your project and remove deprecated calls 2026-05-11T15:51:59,449 or your builds will no longer be supported. 2026-05-11T15:51:59,450 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2026-05-11T15:51:59,451 ******************************************************************************** 2026-05-11T15:51:59,451 !! 2026-05-11T15:51:59,452 corresp(dist, value, root_dir) 2026-05-11T15:51:59,533 running bdist_wheel 2026-05-11T15:51:59,551 running build 2026-05-11T15:51:59,552 running build_py 2026-05-11T15:51:59,558 creating build/lib/lm_eval 2026-05-11T15:51:59,560 copying lm_eval/__main__.py -> build/lib/lm_eval 2026-05-11T15:51:59,562 copying lm_eval/evaluator_utils.py -> build/lib/lm_eval 2026-05-11T15:51:59,565 copying lm_eval/result_schema.py -> build/lib/lm_eval 2026-05-11T15:51:59,567 copying lm_eval/__init__.py -> build/lib/lm_eval 2026-05-11T15:51:59,569 copying lm_eval/evaluator.py -> build/lib/lm_eval 2026-05-11T15:51:59,572 copying lm_eval/defaults.py -> build/lib/lm_eval 2026-05-11T15:51:59,574 copying lm_eval/utils.py -> build/lib/lm_eval 2026-05-11T15:51:59,578 creating build/lib/lm_eval/models 2026-05-11T15:51:59,579 copying lm_eval/models/mamba_lm.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,581 copying lm_eval/models/hf_steered.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,584 copying lm_eval/models/vllm_vlms.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,586 copying lm_eval/models/nemo_lm.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,589 copying lm_eval/models/dummy.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,591 copying lm_eval/models/gguf.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,593 copying lm_eval/models/winml.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,596 copying lm_eval/models/optimum_lm.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,598 copying lm_eval/models/__init__.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,600 copying lm_eval/models/utils_hf.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,602 copying lm_eval/models/vllm_causallms.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,605 copying lm_eval/models/optimum_ipex.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,607 copying lm_eval/models/api_models.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,610 copying lm_eval/models/ibm_watsonx_ai.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,613 copying lm_eval/models/hf_audiolm.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,615 copying lm_eval/models/huggingface.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,619 copying lm_eval/models/openai_completions.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,621 copying lm_eval/models/megatron_lm.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,625 copying lm_eval/models/anthropic_llms.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,627 copying lm_eval/models/textsynth.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,630 copying lm_eval/models/sglang_generate_API.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,632 copying lm_eval/models/hf_vlms.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,634 copying lm_eval/models/utils.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,637 copying lm_eval/models/mistral3.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,639 copying lm_eval/models/sglang_causallms.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,642 copying lm_eval/models/neuron_optimum.py -> build/lib/lm_eval/models 2026-05-11T15:51:59,646 creating build/lib/lm_eval/tasks 2026-05-11T15:51:59,647 copying lm_eval/tasks/_yaml_loader.py -> build/lib/lm_eval/tasks 2026-05-11T15:51:59,650 copying lm_eval/tasks/__init__.py -> build/lib/lm_eval/tasks 2026-05-11T15:51:59,652 copying lm_eval/tasks/_index.py -> build/lib/lm_eval/tasks 2026-05-11T15:51:59,654 copying lm_eval/tasks/manager.py -> build/lib/lm_eval/tasks 2026-05-11T15:51:59,656 copying lm_eval/tasks/_factory.py -> build/lib/lm_eval/tasks 2026-05-11T15:51:59,659 creating build/lib/lm_eval/_cli 2026-05-11T15:51:59,660 copying lm_eval/_cli/subcommand.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,662 copying lm_eval/_cli/validate.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,664 copying lm_eval/_cli/__init__.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,666 copying lm_eval/_cli/harness.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,667 copying lm_eval/_cli/ls.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,669 copying lm_eval/_cli/run.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,672 copying lm_eval/_cli/utils.py -> build/lib/lm_eval/_cli 2026-05-11T15:51:59,674 creating build/lib/lm_eval/api 2026-05-11T15:51:59,675 copying lm_eval/api/group.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,678 copying lm_eval/api/_types.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,680 copying lm_eval/api/filter.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,681 copying lm_eval/api/__init__.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,683 copying lm_eval/api/model.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,685 copying lm_eval/api/_utils.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,687 copying lm_eval/api/registry.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,690 copying lm_eval/api/samplers.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,692 copying lm_eval/api/instance.py -> build/lib/lm_eval/api 2026-05-11T15:51:59,695 creating build/lib/lm_eval/config 2026-05-11T15:51:59,696 copying lm_eval/config/group.py -> build/lib/lm_eval/config 2026-05-11T15:51:59,698 copying lm_eval/config/templates.py -> build/lib/lm_eval/config 2026-05-11T15:51:59,700 copying lm_eval/config/__init__.py -> build/lib/lm_eval/config 2026-05-11T15:51:59,702 copying lm_eval/config/task.py -> build/lib/lm_eval/config 2026-05-11T15:51:59,705 copying lm_eval/config/evaluate_config.py -> build/lib/lm_eval/config 2026-05-11T15:51:59,707 copying lm_eval/config/utils.py -> build/lib/lm_eval/config 2026-05-11T15:51:59,710 creating build/lib/lm_eval/caching 2026-05-11T15:51:59,711 copying lm_eval/caching/__init__.py -> build/lib/lm_eval/caching 2026-05-11T15:51:59,712 copying lm_eval/caching/cache.py -> build/lib/lm_eval/caching 2026-05-11T15:51:59,714 creating build/lib/lm_eval/prompts 2026-05-11T15:51:59,715 copying lm_eval/prompts/__init__.py -> build/lib/lm_eval/prompts 2026-05-11T15:51:59,718 creating build/lib/lm_eval/scorers 2026-05-11T15:51:59,719 copying lm_eval/scorers/_types.py -> build/lib/lm_eval/scorers 2026-05-11T15:51:59,721 copying lm_eval/scorers/__init__.py -> build/lib/lm_eval/scorers 2026-05-11T15:51:59,723 copying lm_eval/scorers/_base.py -> build/lib/lm_eval/scorers 2026-05-11T15:51:59,725 copying lm_eval/scorers/extraction.py -> build/lib/lm_eval/scorers 2026-05-11T15:51:59,728 creating build/lib/lm_eval/filters 2026-05-11T15:51:59,729 copying lm_eval/filters/custom.py -> build/lib/lm_eval/filters 2026-05-11T15:51:59,731 copying lm_eval/filters/decontamination.py -> build/lib/lm_eval/filters 2026-05-11T15:51:59,732 copying lm_eval/filters/__init__.py -> build/lib/lm_eval/filters 2026-05-11T15:51:59,734 copying lm_eval/filters/transformation.py -> build/lib/lm_eval/filters 2026-05-11T15:51:59,736 copying lm_eval/filters/selection.py -> build/lib/lm_eval/filters 2026-05-11T15:51:59,738 copying lm_eval/filters/extraction.py -> build/lib/lm_eval/filters 2026-05-11T15:51:59,741 creating build/lib/lm_eval/loggers 2026-05-11T15:51:59,742 copying lm_eval/loggers/wandb_logger.py -> build/lib/lm_eval/loggers 2026-05-11T15:51:59,744 copying lm_eval/loggers/evaluation_tracker.py -> build/lib/lm_eval/loggers 2026-05-11T15:51:59,747 copying lm_eval/loggers/__init__.py -> build/lib/lm_eval/loggers 2026-05-11T15:51:59,749 copying lm_eval/loggers/utils.py -> build/lib/lm_eval/loggers 2026-05-11T15:51:59,752 creating build/lib/lm_eval/decontamination 2026-05-11T15:51:59,753 copying lm_eval/decontamination/decontaminate.py -> build/lib/lm_eval/decontamination 2026-05-11T15:51:59,755 copying lm_eval/decontamination/janitor.py -> build/lib/lm_eval/decontamination 2026-05-11T15:51:59,757 copying lm_eval/decontamination/__init__.py -> build/lib/lm_eval/decontamination 2026-05-11T15:51:59,759 copying lm_eval/decontamination/archiver.py -> build/lib/lm_eval/decontamination 2026-05-11T15:51:59,761 creating build/lib/lm_eval/tasks/noticia 2026-05-11T15:51:59,762 copying lm_eval/tasks/noticia/utils.py -> build/lib/lm_eval/tasks/noticia 2026-05-11T15:51:59,764 creating build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:51:59,765 copying lm_eval/tasks/arab_culture/prompts.py -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:51:59,767 copying lm_eval/tasks/arab_culture/utils_mcq.py -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:51:59,769 copying lm_eval/tasks/arab_culture/_generate_configs.py -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:51:59,772 creating build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T15:51:59,773 copying lm_eval/tasks/mimic_repsum/utils_perplexity.py -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T15:51:59,775 copying lm_eval/tasks/mimic_repsum/utils.py -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T15:51:59,777 creating build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T15:51:59,778 copying lm_eval/tasks/cnn_dailymail/utils.py -> build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T15:51:59,782 creating build/lib/lm_eval/tasks/humaneval 2026-05-11T15:51:59,783 copying lm_eval/tasks/humaneval/utils.py -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:51:59,787 creating build/lib/lm_eval/tasks/paloma 2026-05-11T15:51:59,788 copying lm_eval/tasks/paloma/paloma_utils.py -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:51:59,790 creating build/lib/lm_eval/tasks/c4 2026-05-11T15:51:59,791 copying lm_eval/tasks/c4/preprocess_c4.py -> build/lib/lm_eval/tasks/c4 2026-05-11T15:51:59,794 creating build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T15:51:59,795 copying lm_eval/tasks/mmlu_prox/lang_libs.py -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T15:51:59,797 copying lm_eval/tasks/mmlu_prox/mmlu_prox_lite_config_generator.py -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T15:51:59,800 copying lm_eval/tasks/mmlu_prox/mmlu_prox_config_generator.py -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T15:51:59,803 creating build/lib/lm_eval/tasks/graphwalks 2026-05-11T15:51:59,804 copying lm_eval/tasks/graphwalks/utils.py -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T15:51:59,807 creating build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:51:59,808 copying lm_eval/tasks/simple_cooccurrence_bias/utils.py -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:51:59,810 creating build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:51:59,811 copying lm_eval/tasks/catalan_bench/utils.py -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:51:59,814 creating build/lib/lm_eval/tasks/winogender 2026-05-11T15:51:59,815 copying lm_eval/tasks/winogender/utils.py -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:51:59,818 creating build/lib/lm_eval/tasks/olaph 2026-05-11T15:51:59,819 copying lm_eval/tasks/olaph/utils_perplexity.py -> build/lib/lm_eval/tasks/olaph 2026-05-11T15:51:59,821 copying lm_eval/tasks/olaph/utils.py -> build/lib/lm_eval/tasks/olaph 2026-05-11T15:51:59,823 creating build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:51:59,824 copying lm_eval/tasks/mediqa_qa2019/utils_perplexity.py -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:51:59,826 copying lm_eval/tasks/mediqa_qa2019/utils.py -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:51:59,829 creating build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:51:59,830 copying lm_eval/tasks/eus_exams/configs.py -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:51:59,832 copying lm_eval/tasks/eus_exams/utils.py -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:51:59,835 creating build/lib/lm_eval/tasks/qasper 2026-05-11T15:51:59,835 copying lm_eval/tasks/qasper/metrics.py -> build/lib/lm_eval/tasks/qasper 2026-05-11T15:51:59,837 copying lm_eval/tasks/qasper/utils.py -> build/lib/lm_eval/tasks/qasper 2026-05-11T15:51:59,840 creating build/lib/lm_eval/tasks/translation 2026-05-11T15:51:59,841 copying lm_eval/tasks/translation/utils.py -> build/lib/lm_eval/tasks/translation 2026-05-11T15:51:59,844 creating build/lib/lm_eval/tasks/mutual 2026-05-11T15:51:59,845 copying lm_eval/tasks/mutual/utils.py -> build/lib/lm_eval/tasks/mutual 2026-05-11T15:51:59,847 creating build/lib/lm_eval/tasks/swde 2026-05-11T15:51:59,848 copying lm_eval/tasks/swde/task.py -> build/lib/lm_eval/tasks/swde 2026-05-11T15:51:59,852 creating build/lib/lm_eval/tasks/belebele 2026-05-11T15:51:59,853 copying lm_eval/tasks/belebele/_generate_configs.py -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:51:59,855 creating build/lib/lm_eval/tasks/winogrande 2026-05-11T15:51:59,856 copying lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/lib/lm_eval/tasks/winogrande 2026-05-11T15:51:59,858 creating build/lib/lm_eval/tasks/chartqa 2026-05-11T15:51:59,859 copying lm_eval/tasks/chartqa/utils.py -> build/lib/lm_eval/tasks/chartqa 2026-05-11T15:51:59,862 creating build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:51:59,863 copying lm_eval/tasks/crows_pairs/utils.py -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:51:59,866 creating build/lib/lm_eval/tasks/kobest 2026-05-11T15:51:59,867 copying lm_eval/tasks/kobest/utils.py -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:51:59,869 creating build/lib/lm_eval/tasks/squad_completion 2026-05-11T15:51:59,870 copying lm_eval/tasks/squad_completion/task.py -> build/lib/lm_eval/tasks/squad_completion 2026-05-11T15:51:59,873 creating build/lib/lm_eval/tasks/wsc273 2026-05-11T15:51:59,874 copying lm_eval/tasks/wsc273/utils.py -> build/lib/lm_eval/tasks/wsc273 2026-05-11T15:51:59,877 creating build/lib/lm_eval/tasks/unitxt 2026-05-11T15:51:59,878 copying lm_eval/tasks/unitxt/task.py -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:51:59,881 creating build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:51:59,882 copying lm_eval/tasks/minerva_math/utils.py -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:51:59,885 creating build/lib/lm_eval/tasks/hellaswag 2026-05-11T15:51:59,886 copying lm_eval/tasks/hellaswag/utils.py -> build/lib/lm_eval/tasks/hellaswag 2026-05-11T15:51:59,889 creating build/lib/lm_eval/tasks/score 2026-05-11T15:51:59,890 copying lm_eval/tasks/score/non_greedy_summarizer.py -> build/lib/lm_eval/tasks/score 2026-05-11T15:51:59,893 copying lm_eval/tasks/score/utils.py -> build/lib/lm_eval/tasks/score 2026-05-11T15:51:59,895 creating build/lib/lm_eval/tasks/realtoxicityprompts 2026-05-11T15:51:59,896 copying lm_eval/tasks/realtoxicityprompts/metric.py -> build/lib/lm_eval/tasks/realtoxicityprompts 2026-05-11T15:51:59,899 creating build/lib/lm_eval/tasks/meqsum 2026-05-11T15:51:59,900 copying lm_eval/tasks/meqsum/utils.py -> build/lib/lm_eval/tasks/meqsum 2026-05-11T15:51:59,904 creating build/lib/lm_eval/tasks/libra 2026-05-11T15:51:59,904 copying lm_eval/tasks/libra/utils.py -> build/lib/lm_eval/tasks/libra 2026-05-11T15:51:59,907 creating build/lib/lm_eval/tasks/common_voice 2026-05-11T15:51:59,908 copying lm_eval/tasks/common_voice/utils.py -> build/lib/lm_eval/tasks/common_voice 2026-05-11T15:51:59,911 creating build/lib/lm_eval/tasks/glianorex 2026-05-11T15:51:59,912 copying lm_eval/tasks/glianorex/preprocess_glianorex.py -> build/lib/lm_eval/tasks/glianorex 2026-05-11T15:51:59,914 creating build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:51:59,915 copying lm_eval/tasks/hendrycks_math/utils.py -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:51:59,918 creating build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:51:59,919 copying lm_eval/tasks/mmlu-pro-plus/utils.py -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:51:59,922 creating build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:51:59,923 copying lm_eval/tasks/arab_culture_completion/prompts.py -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:51:59,925 copying lm_eval/tasks/arab_culture_completion/_generate_configs.py -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:51:59,927 copying lm_eval/tasks/arab_culture_completion/utils_completion.py -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:51:59,930 creating build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:51:59,931 copying lm_eval/tasks/basque_bench/utils.py -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:51:59,933 creating build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:51:59,934 copying lm_eval/tasks/darijammlu/_generate_configs.py -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:51:59,936 copying lm_eval/tasks/darijammlu/utils.py -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:51:59,938 creating build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T15:51:59,939 copying lm_eval/tasks/jsonschema_bench/metrics.py -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T15:51:59,942 creating build/lib/lm_eval/tasks/discrim_eval 2026-05-11T15:51:59,943 copying lm_eval/tasks/discrim_eval/utils.py -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T15:51:59,946 creating build/lib/lm_eval/tasks/toxigen 2026-05-11T15:51:59,947 copying lm_eval/tasks/toxigen/utils.py -> build/lib/lm_eval/tasks/toxigen 2026-05-11T15:51:59,949 creating build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T15:51:59,950 copying lm_eval/tasks/icelandic_winogrande/preprocess_winogrande.py -> build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T15:51:59,952 creating build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T15:51:59,953 copying lm_eval/tasks/darijahellaswag/utils.py -> build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T15:51:59,956 creating build/lib/lm_eval/tasks/truthfulqa 2026-05-11T15:51:59,957 copying lm_eval/tasks/truthfulqa/utils.py -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T15:51:59,960 creating build/lib/lm_eval/tasks/bbq 2026-05-11T15:51:59,961 copying lm_eval/tasks/bbq/utils.py -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:51:59,964 creating build/lib/lm_eval/tasks/bigbench 2026-05-11T15:51:59,965 copying lm_eval/tasks/bigbench/generate_tasks.py -> build/lib/lm_eval/tasks/bigbench 2026-05-11T15:51:59,968 copying lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/lib/lm_eval/tasks/bigbench 2026-05-11T15:51:59,970 creating build/lib/lm_eval/tasks/medqa 2026-05-11T15:51:59,971 copying lm_eval/tasks/medqa/preprocess_medqa.py -> build/lib/lm_eval/tasks/medqa 2026-05-11T15:51:59,974 creating build/lib/lm_eval/tasks/pubmedqa 2026-05-11T15:51:59,975 copying lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/lib/lm_eval/tasks/pubmedqa 2026-05-11T15:51:59,977 creating build/lib/lm_eval/tasks/scrolls 2026-05-11T15:51:59,978 copying lm_eval/tasks/scrolls/task.py -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:51:59,982 creating build/lib/lm_eval/tasks/mlqa 2026-05-11T15:51:59,983 copying lm_eval/tasks/mlqa/generate_tasks.py -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:51:59,985 copying lm_eval/tasks/mlqa/utils.py -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:51:59,988 creating build/lib/lm_eval/tasks/eus_trivia 2026-05-11T15:51:59,989 copying lm_eval/tasks/eus_trivia/utils.py -> build/lib/lm_eval/tasks/eus_trivia 2026-05-11T15:51:59,992 creating build/lib/lm_eval/tasks/fda 2026-05-11T15:51:59,993 copying lm_eval/tasks/fda/task.py -> build/lib/lm_eval/tasks/fda 2026-05-11T15:51:59,995 creating build/lib/lm_eval/tasks/coqa 2026-05-11T15:51:59,996 copying lm_eval/tasks/coqa/utils.py -> build/lib/lm_eval/tasks/coqa 2026-05-11T15:51:59,999 creating build/lib/lm_eval/tasks/medmcqa 2026-05-11T15:52:00,000 copying lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/lib/lm_eval/tasks/medmcqa 2026-05-11T15:52:00,003 creating build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,004 copying lm_eval/tasks/ruler/prepare_niah.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,006 copying lm_eval/tasks/ruler/niah_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,009 copying lm_eval/tasks/ruler/qa_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,011 copying lm_eval/tasks/ruler/essays.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,013 copying lm_eval/tasks/ruler/vt_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,015 copying lm_eval/tasks/ruler/fwe_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,017 copying lm_eval/tasks/ruler/common_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,019 copying lm_eval/tasks/ruler/cwe_utils.py -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:00,022 creating build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:00,023 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:00,025 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:00,027 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:00,029 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.py -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:00,031 creating build/lib/lm_eval/tasks/wikitext 2026-05-11T15:52:00,032 copying lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/lib/lm_eval/tasks/wikitext 2026-05-11T15:52:00,035 creating build/lib/lm_eval/tasks/med_text_classification 2026-05-11T15:52:00,036 copying lm_eval/tasks/med_text_classification/utils.py -> build/lib/lm_eval/tasks/med_text_classification 2026-05-11T15:52:00,039 creating build/lib/lm_eval/tasks/bbh 2026-05-11T15:52:00,040 copying lm_eval/tasks/bbh/_generate_configs.py -> build/lib/lm_eval/tasks/bbh 2026-05-11T15:52:00,042 creating build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:00,043 copying lm_eval/tasks/csatqa/_generate_configs.py -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:00,045 copying lm_eval/tasks/csatqa/utils.py -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:00,047 creating build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:00,049 copying lm_eval/tasks/xnli/utils.py -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:00,053 creating build/lib/lm_eval/tasks/eus_reading 2026-05-11T15:52:00,054 copying lm_eval/tasks/eus_reading/utils.py -> build/lib/lm_eval/tasks/eus_reading 2026-05-11T15:52:00,056 creating build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:00,057 copying lm_eval/tasks/longbench/_generate_config.py -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:00,060 copying lm_eval/tasks/longbench/metrics.py -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:00,063 copying lm_eval/tasks/longbench/utils.py -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:00,065 creating build/lib/lm_eval/tasks/mmlu 2026-05-11T15:52:00,066 copying lm_eval/tasks/mmlu/_generate_configs.py -> build/lib/lm_eval/tasks/mmlu 2026-05-11T15:52:00,070 creating build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:00,071 copying lm_eval/tasks/blimp/generate_configs.py -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:00,074 creating build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:00,075 copying lm_eval/tasks/mmmu/utils.py -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:00,078 creating build/lib/lm_eval/tasks/histoires_morales 2026-05-11T15:52:00,079 copying lm_eval/tasks/histoires_morales/utils.py -> build/lib/lm_eval/tasks/histoires_morales 2026-05-11T15:52:00,084 creating build/lib/lm_eval/tasks/afrimmlu 2026-05-11T15:52:00,085 copying lm_eval/tasks/afrimmlu/gen_utils.py -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T15:52:00,087 copying lm_eval/tasks/afrimmlu/utils.py -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T15:52:00,089 creating build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:00,090 copying lm_eval/tasks/mbpp/utils.py -> build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:00,093 creating build/lib/lm_eval/tasks/moral_stories 2026-05-11T15:52:00,094 copying lm_eval/tasks/moral_stories/utils.py -> build/lib/lm_eval/tasks/moral_stories 2026-05-11T15:52:00,096 creating build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:00,097 copying lm_eval/tasks/lingoly/utils.py -> build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:00,099 copying lm_eval/tasks/lingoly/script.py -> build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:00,102 creating build/lib/lm_eval/tasks/logiqa2 2026-05-11T15:52:00,103 copying lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T15:52:00,107 creating build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:00,108 copying lm_eval/tasks/mmlu_pro/utils.py -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:00,111 creating build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:00,112 copying lm_eval/tasks/ceval/_generate_configs.py -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:00,115 creating build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:00,116 copying lm_eval/tasks/aclue/_generate_configs.py -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:00,118 creating build/lib/lm_eval/tasks/webqs 2026-05-11T15:52:00,119 copying lm_eval/tasks/webqs/utils.py -> build/lib/lm_eval/tasks/webqs 2026-05-11T15:52:00,122 creating build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:00,123 copying lm_eval/tasks/hendrycks_ethics/utils.py -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:00,125 creating build/lib/lm_eval/tasks/afrimgsm 2026-05-11T15:52:00,126 copying lm_eval/tasks/afrimgsm/gen_utils.py -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T15:52:00,128 copying lm_eval/tasks/afrimgsm/utils.py -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T15:52:00,133 creating build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:00,134 copying lm_eval/tasks/xwinograd/utils.py -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:00,136 creating build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T15:52:00,137 copying lm_eval/tasks/med_prescriptions/utils.py -> build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T15:52:00,142 creating build/lib/lm_eval/tasks/afrixnli 2026-05-11T15:52:00,143 copying lm_eval/tasks/afrixnli/gen_utils.py -> build/lib/lm_eval/tasks/afrixnli 2026-05-11T15:52:00,145 copying lm_eval/tasks/afrixnli/utils.py -> build/lib/lm_eval/tasks/afrixnli 2026-05-11T15:52:00,148 creating build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:00,149 copying lm_eval/tasks/agieval/utils.py -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:00,153 creating build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:00,154 copying lm_eval/tasks/ifeval/instructions_registry.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:00,156 copying lm_eval/tasks/ifeval/instructions_util.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:00,159 copying lm_eval/tasks/ifeval/instructions.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:00,163 copying lm_eval/tasks/ifeval/utils.py -> build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:00,165 creating build/lib/lm_eval/tasks/mgsm 2026-05-11T15:52:00,166 copying lm_eval/tasks/mgsm/utils.py -> build/lib/lm_eval/tasks/mgsm 2026-05-11T15:52:00,171 creating build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:00,172 copying lm_eval/tasks/tinyBenchmarks/agg_functions.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:00,174 copying lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:00,176 copying lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:00,178 copying lm_eval/tasks/tinyBenchmarks/utils_winogrande.py -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:00,181 creating build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:00,182 copying lm_eval/tasks/humaneval_infilling/utils.py -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:00,184 creating build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:00,185 copying lm_eval/tasks/arabicmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:00,188 copying lm_eval/tasks/arabicmmlu/utils.py -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:00,191 creating build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:00,192 copying lm_eval/tasks/french_bench/preprocess_wikitext.py -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:00,194 copying lm_eval/tasks/french_bench/utils.py -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:00,197 creating build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:00,198 copying lm_eval/tasks/egymmlu/_generate_configs.py -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:00,200 copying lm_eval/tasks/egymmlu/utils.py -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:00,203 creating build/lib/lm_eval/tasks/drop 2026-05-11T15:52:00,204 copying lm_eval/tasks/drop/utils.py -> build/lib/lm_eval/tasks/drop 2026-05-11T15:52:00,207 creating build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:00,209 copying lm_eval/tasks/cmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:00,211 creating build/lib/lm_eval/tasks/logiqa 2026-05-11T15:52:00,212 copying lm_eval/tasks/logiqa/utils_logiqa.py -> build/lib/lm_eval/tasks/logiqa 2026-05-11T15:52:00,215 creating build/lib/lm_eval/tasks/race 2026-05-11T15:52:00,216 copying lm_eval/tasks/race/preprocess_race.py -> build/lib/lm_eval/tasks/race 2026-05-11T15:52:00,218 creating build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:00,219 copying lm_eval/tasks/spanish_bench/utils.py -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:00,222 creating build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:00,223 copying lm_eval/tasks/slr_bench/lm_eval_slr_bench.py -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:00,226 creating build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:00,227 copying lm_eval/tasks/truthfulqa-multi/utils.py -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:00,230 creating build/lib/lm_eval/tasks/aime 2026-05-11T15:52:00,231 copying lm_eval/tasks/aime/utils.py -> build/lib/lm_eval/tasks/aime 2026-05-11T15:52:00,234 creating build/lib/lm_eval/tasks/mmlusr 2026-05-11T15:52:00,235 copying lm_eval/tasks/mmlusr/config.py -> build/lib/lm_eval/tasks/mmlusr 2026-05-11T15:52:00,239 creating build/lib/lm_eval/tasks/squadv2 2026-05-11T15:52:00,240 copying lm_eval/tasks/squadv2/task.py -> build/lib/lm_eval/tasks/squadv2 2026-05-11T15:52:00,243 creating build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:00,244 copying lm_eval/tasks/galician_bench/utils.py -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:00,247 creating build/lib/lm_eval/tasks/eq_bench 2026-05-11T15:52:00,248 copying lm_eval/tasks/eq_bench/utils.py -> build/lib/lm_eval/tasks/eq_bench 2026-05-11T15:52:00,251 creating build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:00,252 copying lm_eval/tasks/basqueglue/utils.py -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:00,256 creating build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T15:52:00,257 copying lm_eval/tasks/egyhellaswag/utils.py -> build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T15:52:00,260 creating build/lib/lm_eval/tasks/wmt2016 2026-05-11T15:52:00,261 copying lm_eval/tasks/wmt2016/metrics.py -> build/lib/lm_eval/tasks/wmt2016 2026-05-11T15:52:00,263 creating build/lib/lm_eval/tasks/mathqa 2026-05-11T15:52:00,264 copying lm_eval/tasks/mathqa/utils.py -> build/lib/lm_eval/tasks/mathqa 2026-05-11T15:52:00,267 creating build/lib/lm_eval/tasks/qa4mre 2026-05-11T15:52:00,268 copying lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T15:52:00,270 creating build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:00,271 copying lm_eval/tasks/pisa/utils.py -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:00,275 creating build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:00,276 copying lm_eval/tasks/esbbq/utils.py -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:00,279 creating build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:00,280 copying lm_eval/tasks/med_concepts_qa/_generate_configs.py -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:00,283 creating build/lib/lm_eval/tasks/groundcocoa 2026-05-11T15:52:00,284 copying lm_eval/tasks/groundcocoa/utils.py -> build/lib/lm_eval/tasks/groundcocoa 2026-05-11T15:52:00,287 creating build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:00,288 copying lm_eval/tasks/evalita_llm/sum_utils.py -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:00,290 copying lm_eval/tasks/evalita_llm/metrics.py -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:00,292 copying lm_eval/tasks/evalita_llm/utils.py -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:00,295 creating build/lib/lm_eval/tasks/medtext 2026-05-11T15:52:00,296 copying lm_eval/tasks/medtext/utils_perplexity.py -> build/lib/lm_eval/tasks/medtext 2026-05-11T15:52:00,298 copying lm_eval/tasks/medtext/utils.py -> build/lib/lm_eval/tasks/medtext 2026-05-11T15:52:00,301 creating build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:00,302 copying lm_eval/tasks/xquad/utils.py -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:00,304 creating build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:00,305 copying lm_eval/tasks/paws-x/_generate_config.py -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:00,307 copying lm_eval/tasks/paws-x/utils.py -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:00,310 creating build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:00,311 copying lm_eval/tasks/xcopa/utils.py -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:00,315 creating build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T15:52:00,317 copying lm_eval/tasks/openai-mmmlu/_generate_configs.py -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T15:52:00,320 creating build/lib/lm_eval/tasks/mts_dialog 2026-05-11T15:52:00,321 copying lm_eval/tasks/mts_dialog/utils_perplexity.py -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T15:52:00,323 copying lm_eval/tasks/mts_dialog/utils.py -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T15:52:00,325 creating build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:00,326 copying lm_eval/tasks/meddialog/utils_perplexity.py -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:00,328 copying lm_eval/tasks/meddialog/utils.py -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:00,331 creating build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:00,332 copying lm_eval/tasks/cabbq/utils.py -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:00,336 creating build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:00,337 copying lm_eval/tasks/babilong/common_utils.py -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:00,339 creating build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:00,340 copying lm_eval/tasks/careqa/utils_open.py -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:00,342 copying lm_eval/tasks/careqa/utils_perplexity.py -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:00,344 copying lm_eval/tasks/careqa/utils.py -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:00,347 creating build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:00,348 copying lm_eval/tasks/metabench/process_docs.py -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:00,350 copying lm_eval/tasks/metabench/process_docs_permute.py -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:00,354 creating build/lib/lm_eval/tasks/copal_id 2026-05-11T15:52:00,355 copying lm_eval/tasks/copal_id/utils.py -> build/lib/lm_eval/tasks/copal_id 2026-05-11T15:52:00,358 creating build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:00,359 copying lm_eval/tasks/tmmluplus/default/_generate_configs.py -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:00,362 copying lm_eval/tasks/tmmluplus/default/utils.py -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:00,364 creating build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:00,366 copying lm_eval/tasks/aradice/winogrande/metrics.py -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:00,368 copying lm_eval/tasks/aradice/winogrande/utils.py -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:00,371 creating build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:00,372 copying lm_eval/tasks/aradice/openbookqa/metrics.py -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:00,374 copying lm_eval/tasks/aradice/openbookqa/utils.py -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:00,376 creating build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:00,377 copying lm_eval/tasks/aradice/cultural-benchmark/metrics.py -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:00,379 copying lm_eval/tasks/aradice/cultural-benchmark/utils.py -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:00,382 creating build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:00,383 copying lm_eval/tasks/aradice/piqa/metrics.py -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:00,385 creating build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:00,386 copying lm_eval/tasks/aradice/truthfulqa_mcq/metrics.py -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:00,389 creating build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:00,390 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/metrics.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:00,392 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/utils.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:00,395 creating build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:00,396 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/metrics.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:00,398 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/utils.py -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:00,401 creating build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:00,402 copying lm_eval/tasks/aradice/boolq/MSA/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:00,404 copying lm_eval/tasks/aradice/boolq/MSA/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:00,406 creating build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:00,407 copying lm_eval/tasks/aradice/boolq/LEV/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:00,410 copying lm_eval/tasks/aradice/boolq/LEV/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:00,412 creating build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:00,413 copying lm_eval/tasks/aradice/boolq/EGY/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:00,415 copying lm_eval/tasks/aradice/boolq/EGY/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:00,417 creating build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:00,419 copying lm_eval/tasks/aradice/boolq/ENG/metrics.py -> build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:00,421 copying lm_eval/tasks/aradice/boolq/ENG/utils.py -> build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:00,424 creating build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:00,426 copying lm_eval/tasks/e2lmc/noor/_generate_configs.py -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:00,429 creating build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:00,430 copying lm_eval/tasks/e2lmc/mmlu_early_training/custom_metrics.py -> build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:00,433 creating build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:00,434 copying lm_eval/tasks/mmlu_prox/mr/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:00,436 creating build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:00,437 copying lm_eval/tasks/mmlu_prox/de/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:00,440 creating build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:00,441 copying lm_eval/tasks/mmlu_prox/yo/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:00,444 creating build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:00,445 copying lm_eval/tasks/mmlu_prox/ru/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:00,447 creating build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:00,448 copying lm_eval/tasks/mmlu_prox/zh/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:00,450 creating build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:00,451 copying lm_eval/tasks/mmlu_prox/af/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:00,454 creating build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:00,455 copying lm_eval/tasks/mmlu_prox/th/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:00,457 creating build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:00,458 copying lm_eval/tasks/mmlu_prox/zu/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:00,461 creating build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:00,462 copying lm_eval/tasks/mmlu_prox/ne/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:00,464 creating build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:00,465 copying lm_eval/tasks/mmlu_prox/vi/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:00,468 creating build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:00,469 copying lm_eval/tasks/mmlu_prox/ko/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:00,471 creating build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:00,472 copying lm_eval/tasks/mmlu_prox/bn/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:00,475 creating build/lib/lm_eval/tasks/mmlu_prox/template 2026-05-11T15:52:00,476 copying lm_eval/tasks/mmlu_prox/template/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/template 2026-05-11T15:52:00,478 creating build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:00,479 copying lm_eval/tasks/mmlu_prox/en/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:00,482 creating build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:00,483 copying lm_eval/tasks/mmlu_prox/uk/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:00,486 creating build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:00,487 copying lm_eval/tasks/mmlu_prox/sw/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:00,490 creating build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:00,491 copying lm_eval/tasks/mmlu_prox/es/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:00,493 creating build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:00,494 copying lm_eval/tasks/mmlu_prox/te/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:00,497 creating build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:00,498 copying lm_eval/tasks/mmlu_prox/ja/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:00,500 creating build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:00,501 copying lm_eval/tasks/mmlu_prox/hi/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:00,504 creating build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:00,505 copying lm_eval/tasks/mmlu_prox/it/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:00,507 creating build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:00,509 copying lm_eval/tasks/mmlu_prox/sr/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:00,511 creating build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:00,513 copying lm_eval/tasks/mmlu_prox/wo/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:00,515 creating build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:00,516 copying lm_eval/tasks/mmlu_prox/fr/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:00,519 creating build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:00,520 copying lm_eval/tasks/mmlu_prox/cs/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:00,522 creating build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:00,523 copying lm_eval/tasks/mmlu_prox/pt/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:00,526 creating build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:00,527 copying lm_eval/tasks/mmlu_prox/id/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:00,530 creating build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:00,531 copying lm_eval/tasks/mmlu_prox/ar/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:00,533 creating build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:00,534 copying lm_eval/tasks/mmlu_prox/ur/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:00,537 creating build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:00,538 copying lm_eval/tasks/mmlu_prox/hu/utils.py -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:00,541 creating build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T15:52:00,542 copying lm_eval/tasks/catalan_bench/truthfulqa_va/utils.py -> build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T15:52:00,545 creating build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:00,546 copying lm_eval/tasks/catalan_bench/flores_ca/create_yamls_flores_ca.py -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:00,549 creating build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:00,550 copying lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:00,553 creating build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:00,554 copying lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:00,557 creating build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:00,558 copying lm_eval/tasks/okapi/arc_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:00,560 creating build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:00,561 copying lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:00,566 creating build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:00,567 copying lm_eval/tasks/click/click_lang/utils.py -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:00,570 creating build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:00,571 copying lm_eval/tasks/click/click_cul/utils.py -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:00,574 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:00,576 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:00,579 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:00,580 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:00,582 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:00,583 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:00,586 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:00,587 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:00,590 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:00,591 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:00,593 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:00,594 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:00,597 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:00,598 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:00,600 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:00,601 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:00,603 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:00,604 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:00,607 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:00,608 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:00,610 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:00,611 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:00,614 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:00,615 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:00,617 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:00,619 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:00,621 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:00,622 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:00,624 creating build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:00,626 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:00,628 creating build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:00,629 copying lm_eval/tasks/score/agi_eval/utils_agieval.py -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:00,632 creating build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:00,633 copying lm_eval/tasks/score/math/math_grader.py -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:00,636 copying lm_eval/tasks/score/math/utils_math.py -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:00,639 creating build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:52:00,640 copying lm_eval/tasks/score/mmlu_pro/utils_mmlu_pro.py -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:52:00,643 creating build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:00,645 copying lm_eval/tasks/acpbench/gen_2shot/acp_utils.py -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:00,648 creating build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:00,649 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_utils.py -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:00,653 creating build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:00,654 copying lm_eval/tasks/basque_bench/flores_eu/create_yamls_flores_eu.py -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:00,656 creating build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T15:52:00,658 copying lm_eval/tasks/super_glue/cb/aggregate.py -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T15:52:00,660 copying lm_eval/tasks/super_glue/cb/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T15:52:00,662 creating build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T15:52:00,663 copying lm_eval/tasks/super_glue/copa/utils.py -> build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T15:52:00,666 creating build/lib/lm_eval/tasks/super_glue/record 2026-05-11T15:52:00,667 copying lm_eval/tasks/super_glue/record/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T15:52:00,669 copying lm_eval/tasks/super_glue/record/util.py -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T15:52:00,671 creating build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T15:52:00,672 copying lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T15:52:00,675 creating build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T15:52:00,676 copying lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T15:52:00,678 copying lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T15:52:00,681 creating build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:00,683 copying lm_eval/tasks/hrm8k/en/utils.py -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:00,685 creating build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:00,687 copying lm_eval/tasks/hrm8k/default/utils.py -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:00,691 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:52:00,693 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:52:00,695 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:52:00,696 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:52:00,699 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:00,700 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:00,702 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:52:00,703 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:52:00,706 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:52:00,707 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:52:00,710 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:52:00,710 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:52:00,713 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:52:00,714 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:52:00,716 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:52:00,717 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:52:00,720 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:00,720 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:00,723 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:52:00,724 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:52:00,726 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:52:00,727 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:52:00,730 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:52:00,731 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:52:00,733 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:52:00,734 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:52:00,737 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:52:00,738 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:52:00,740 creating build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:00,741 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/utils.py -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:00,744 creating build/lib/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T15:52:00,746 copying lm_eval/tasks/noreval/noropenbookqa/utils.py -> build/lib/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T15:52:00,749 creating build/lib/lm_eval/tasks/noreval/norec 2026-05-11T15:52:00,750 copying lm_eval/tasks/noreval/norec/utils.py -> build/lib/lm_eval/tasks/noreval/norec 2026-05-11T15:52:00,752 creating build/lib/lm_eval/tasks/noreval/noridiom 2026-05-11T15:52:00,753 copying lm_eval/tasks/noreval/noridiom/utils.py -> build/lib/lm_eval/tasks/noreval/noridiom 2026-05-11T15:52:00,756 creating build/lib/lm_eval/tasks/noreval/norsumm 2026-05-11T15:52:00,757 copying lm_eval/tasks/noreval/norsumm/utils.py -> build/lib/lm_eval/tasks/noreval/norsumm 2026-05-11T15:52:00,760 creating build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:00,761 copying lm_eval/tasks/noreval/norquad/utils.py -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:00,765 creating build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:00,766 copying lm_eval/tasks/noreval/ask_gec/errant.py -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:00,771 creating build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:00,773 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/utils.py -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:00,775 creating build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:00,776 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/utils.py -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:00,780 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T15:52:00,782 copying lm_eval/tasks/noreval/nortruthfulqa/generation/utils.py -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T15:52:00,784 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:00,786 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/utils.py -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:00,789 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:00,790 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/utils.py -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:00,795 creating build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:00,797 copying lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:00,800 creating build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:00,801 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:00,805 creating build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:00,806 copying lm_eval/tasks/bbh/cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:00,809 creating build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:00,810 copying lm_eval/tasks/bbh/zeroshot/utils.py -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:00,813 creating build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:00,814 copying lm_eval/tasks/global_piqa/completions/_generate_config.py -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:00,817 creating build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:00,818 copying lm_eval/tasks/global_piqa/prompted/_generate_config.py -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:00,822 creating build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:00,823 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:00,827 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:00,829 copying lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:00,833 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:00,835 copying lm_eval/tasks/afrimmlu/translate/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:00,837 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:00,838 copying lm_eval/tasks/afrimmlu/translate/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:00,841 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:00,842 copying lm_eval/tasks/afrimmlu/translate/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:00,844 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:00,845 copying lm_eval/tasks/afrimmlu/translate/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:00,848 creating build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:00,849 copying lm_eval/tasks/afrimmlu/translate/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:00,851 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:00,853 copying lm_eval/tasks/afrimmlu/direct/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:00,855 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:00,856 copying lm_eval/tasks/afrimmlu/direct/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:00,858 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:00,859 copying lm_eval/tasks/afrimmlu/direct/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:00,862 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:00,863 copying lm_eval/tasks/afrimmlu/direct/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:00,865 creating build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:00,866 copying lm_eval/tasks/afrimmlu/direct/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:00,868 creating build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:00,870 copying lm_eval/tasks/darija_bench/darija_transliteration/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:00,873 creating build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:00,874 copying lm_eval/tasks/darija_bench/darija_sentiment/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:00,876 creating build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:00,877 copying lm_eval/tasks/darija_bench/darija_translation/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:00,880 creating build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:52:00,881 copying lm_eval/tasks/darija_bench/darija_summarization/utils.py -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:52:00,884 creating build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:00,886 copying lm_eval/tasks/leaderboard/math/utils.py -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:00,889 creating build/lib/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T15:52:00,890 copying lm_eval/tasks/leaderboard/mmlu_pro/utils.py -> build/lib/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T15:52:00,893 creating build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:00,894 copying lm_eval/tasks/leaderboard/ifeval/instructions_registry.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:00,896 copying lm_eval/tasks/leaderboard/ifeval/instructions_util.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:00,899 copying lm_eval/tasks/leaderboard/ifeval/instructions.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:00,903 copying lm_eval/tasks/leaderboard/ifeval/utils.py -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:00,905 creating build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:00,906 copying lm_eval/tasks/leaderboard/musr/utils.py -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:00,909 creating build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:00,910 copying lm_eval/tasks/leaderboard/gpqa/utils.py -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:00,924 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:00,926 copying lm_eval/tasks/afrixnli/translate/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:00,928 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:00,929 copying lm_eval/tasks/afrixnli/translate/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:00,932 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:00,933 copying lm_eval/tasks/afrixnli/translate/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:00,935 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:00,936 copying lm_eval/tasks/afrixnli/translate/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:00,938 creating build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:00,939 copying lm_eval/tasks/afrixnli/translate/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:00,942 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:00,944 copying lm_eval/tasks/afrixnli/direct/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:00,946 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:00,947 copying lm_eval/tasks/afrixnli/direct/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:00,949 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:00,950 copying lm_eval/tasks/afrixnli/direct/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:00,953 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:00,954 copying lm_eval/tasks/afrixnli/direct/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:00,956 creating build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:00,957 copying lm_eval/tasks/afrixnli/direct/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:00,960 creating build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:00,962 copying lm_eval/tasks/afrixnli/lai prompt/translate/utils.py -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:00,964 creating build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:00,965 copying lm_eval/tasks/afrixnli/lai prompt/direct/utils.py -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:00,968 creating build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:00,969 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/utils.py -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:00,972 creating build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:00,973 copying lm_eval/tasks/afrixnli/anli prompt/translate/utils.py -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:00,975 creating build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:00,976 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:00,979 creating build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T15:52:00,980 copying lm_eval/tasks/ifeval/multilingual/instructions_registry.py -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T15:52:00,982 copying lm_eval/tasks/ifeval/multilingual/utils.py -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T15:52:00,985 creating build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:52:00,986 copying lm_eval/tasks/ifeval/multilingual/instructions/__init__.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:52:00,988 copying lm_eval/tasks/ifeval/multilingual/instructions/es_instructions.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:52:00,992 copying lm_eval/tasks/ifeval/multilingual/instructions/ca_instructions.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:52:00,995 creating build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:52:00,996 copying lm_eval/tasks/ifeval/multilingual/instruction_utils/es_instructions_util.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:52:00,999 copying lm_eval/tasks/ifeval/multilingual/instruction_utils/__init__.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:52:01,001 copying lm_eval/tasks/ifeval/multilingual/instruction_utils/ca_instructions_util.py -> build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:52:01,006 creating build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:01,008 copying lm_eval/tasks/global_mmlu/default/de/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:01,010 creating build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:01,011 copying lm_eval/tasks/global_mmlu/default/yo/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:01,014 creating build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:01,015 copying lm_eval/tasks/global_mmlu/default/zh/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:01,017 creating build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:01,018 copying lm_eval/tasks/global_mmlu/default/ko/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:01,021 creating build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:01,022 copying lm_eval/tasks/global_mmlu/default/bn/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:01,024 creating build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:01,025 copying lm_eval/tasks/global_mmlu/default/en/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:01,027 creating build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:01,028 copying lm_eval/tasks/global_mmlu/default/sw/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:01,031 creating build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:01,032 copying lm_eval/tasks/global_mmlu/default/es/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:01,034 creating build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:01,035 copying lm_eval/tasks/global_mmlu/default/ja/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:01,038 creating build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:01,039 copying lm_eval/tasks/global_mmlu/default/hi/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:01,041 creating build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:01,042 copying lm_eval/tasks/global_mmlu/default/it/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:01,045 creating build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:01,046 copying lm_eval/tasks/global_mmlu/default/fr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:01,048 creating build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:01,049 copying lm_eval/tasks/global_mmlu/default/pt/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:01,052 creating build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:01,053 copying lm_eval/tasks/global_mmlu/default/id/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:01,055 creating build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:01,056 copying lm_eval/tasks/global_mmlu/default/ar/utils.py -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:01,059 creating build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:01,061 copying lm_eval/tasks/global_mmlu/full/ha/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:01,063 creating build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:01,064 copying lm_eval/tasks/global_mmlu/full/ky/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:01,067 creating build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:01,068 copying lm_eval/tasks/global_mmlu/full/de/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:01,071 creating build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:01,072 copying lm_eval/tasks/global_mmlu/full/so/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:01,074 creating build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:01,075 copying lm_eval/tasks/global_mmlu/full/yo/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:01,078 creating build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:01,079 copying lm_eval/tasks/global_mmlu/full/ru/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:01,082 creating build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:01,083 copying lm_eval/tasks/global_mmlu/full/zh/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:01,086 creating build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:01,087 copying lm_eval/tasks/global_mmlu/full/nl/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:01,089 creating build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:01,090 copying lm_eval/tasks/global_mmlu/full/si/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:01,093 creating build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:01,094 copying lm_eval/tasks/global_mmlu/full/ne/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:01,097 creating build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:01,098 copying lm_eval/tasks/global_mmlu/full/vi/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:01,101 creating build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:01,102 copying lm_eval/tasks/global_mmlu/full/ig/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:01,104 creating build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:01,106 copying lm_eval/tasks/global_mmlu/full/ro/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:01,108 creating build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:01,110 copying lm_eval/tasks/global_mmlu/full/ko/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:01,112 creating build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:01,113 copying lm_eval/tasks/global_mmlu/full/lt/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:01,116 creating build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:01,117 copying lm_eval/tasks/global_mmlu/full/pl/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:01,120 creating build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:01,121 copying lm_eval/tasks/global_mmlu/full/sn/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:01,124 creating build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:01,125 copying lm_eval/tasks/global_mmlu/full/bn/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:01,127 creating build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:01,129 copying lm_eval/tasks/global_mmlu/full/ny/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:01,131 creating build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:01,132 copying lm_eval/tasks/global_mmlu/full/en/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:01,135 creating build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:01,136 copying lm_eval/tasks/global_mmlu/full/uk/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:01,139 creating build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:01,140 copying lm_eval/tasks/global_mmlu/full/sw/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:01,142 creating build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:01,143 copying lm_eval/tasks/global_mmlu/full/es/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:01,146 creating build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:01,147 copying lm_eval/tasks/global_mmlu/full/te/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:01,150 creating build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:01,151 copying lm_eval/tasks/global_mmlu/full/fil/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:01,153 creating build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:01,154 copying lm_eval/tasks/global_mmlu/full/ja/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:01,157 creating build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:01,158 copying lm_eval/tasks/global_mmlu/full/hi/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:01,161 creating build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:01,162 copying lm_eval/tasks/global_mmlu/full/ms/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:01,165 creating build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:01,166 copying lm_eval/tasks/global_mmlu/full/it/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:01,168 creating build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:01,169 copying lm_eval/tasks/global_mmlu/full/sr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:01,172 creating build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:01,173 copying lm_eval/tasks/global_mmlu/full/mg/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:01,176 creating build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:01,177 copying lm_eval/tasks/global_mmlu/full/sv/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:01,180 creating build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:01,181 copying lm_eval/tasks/global_mmlu/full/fr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:01,183 creating build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:01,185 copying lm_eval/tasks/global_mmlu/full/tr/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:01,188 creating build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:01,189 copying lm_eval/tasks/global_mmlu/full/el/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:01,192 creating build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:01,193 copying lm_eval/tasks/global_mmlu/full/he/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:01,195 creating build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:01,196 copying lm_eval/tasks/global_mmlu/full/cs/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:01,199 creating build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:01,200 copying lm_eval/tasks/global_mmlu/full/pt/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:01,203 creating build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:01,204 copying lm_eval/tasks/global_mmlu/full/id/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:01,207 creating build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:01,208 copying lm_eval/tasks/global_mmlu/full/ar/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:01,210 creating build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:01,211 copying lm_eval/tasks/global_mmlu/full/fa/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:01,214 creating build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:01,215 copying lm_eval/tasks/global_mmlu/full/am/utils.py -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:01,218 creating build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:01,219 copying lm_eval/tasks/tmlu/default/_generate_configs.py -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:01,222 copying lm_eval/tasks/tmlu/default/utils.py -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:01,224 creating build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:01,225 copying lm_eval/tasks/spanish_bench/flores_es/create_yamls_flores_es.py -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:01,230 creating build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:01,232 copying lm_eval/tasks/include/few_shot_en/Indonesian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:01,234 creating build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:01,235 copying lm_eval/tasks/include/few_shot_en/Bulgarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:01,237 creating build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:01,239 copying lm_eval/tasks/include/few_shot_en/Bengali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:01,241 creating build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:01,242 copying lm_eval/tasks/include/few_shot_en/Turkish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:01,244 creating build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:01,245 copying lm_eval/tasks/include/few_shot_en/Belarusian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:01,248 creating build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:01,249 copying lm_eval/tasks/include/few_shot_en/Greek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:01,251 creating build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:01,252 copying lm_eval/tasks/include/few_shot_en/Hungarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:01,254 creating build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:01,255 copying lm_eval/tasks/include/few_shot_en/Dutch/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:01,258 creating build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:52:01,259 copying lm_eval/tasks/include/few_shot_en/Korean/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:52:01,261 creating build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:01,262 copying lm_eval/tasks/include/few_shot_en/German/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:01,264 creating build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:01,265 copying lm_eval/tasks/include/few_shot_en/Polish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:01,268 creating build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:01,269 copying lm_eval/tasks/include/few_shot_en/Urdu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:01,271 creating build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:01,272 copying lm_eval/tasks/include/few_shot_en/North Macedonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:01,274 creating build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:01,276 copying lm_eval/tasks/include/few_shot_en/Ukrainian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:01,278 creating build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:01,279 copying lm_eval/tasks/include/few_shot_en/Chinese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:01,282 creating build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:01,282 copying lm_eval/tasks/include/few_shot_en/Portuguese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:01,285 creating build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:52:01,286 copying lm_eval/tasks/include/few_shot_en/Georgian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:52:01,288 creating build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:01,289 copying lm_eval/tasks/include/few_shot_en/French/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:01,291 creating build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:01,292 copying lm_eval/tasks/include/few_shot_en/Spanish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:01,294 creating build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:01,296 copying lm_eval/tasks/include/few_shot_en/Serbian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:01,298 creating build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:01,299 copying lm_eval/tasks/include/few_shot_en/Malay/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:01,301 creating build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:01,302 copying lm_eval/tasks/include/few_shot_en/Arabic/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:01,305 creating build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:01,306 copying lm_eval/tasks/include/few_shot_en/Vietnamese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:01,308 creating build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:01,309 copying lm_eval/tasks/include/few_shot_en/Russian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:01,311 creating build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:01,312 copying lm_eval/tasks/include/few_shot_en/Japanese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:01,315 creating build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:01,316 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:01,318 creating build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:01,319 copying lm_eval/tasks/include/few_shot_en/Persian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:01,322 creating build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:01,323 copying lm_eval/tasks/include/few_shot_en/Albanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:01,325 creating build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:01,327 copying lm_eval/tasks/include/few_shot_en/Croatian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:01,329 creating build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:52:01,330 copying lm_eval/tasks/include/few_shot_en/Nepali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:52:01,333 creating build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:01,334 copying lm_eval/tasks/include/few_shot_en/Italian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:01,336 creating build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:52:01,337 copying lm_eval/tasks/include/few_shot_en/Tamil/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:52:01,339 creating build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:52:01,340 copying lm_eval/tasks/include/few_shot_en/Basque/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:52:01,343 creating build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:52:01,344 copying lm_eval/tasks/include/few_shot_en/Hebrew/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:52:01,346 creating build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:01,347 copying lm_eval/tasks/include/few_shot_en/Estonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:01,350 creating build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:01,351 copying lm_eval/tasks/include/few_shot_en/Telugu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:01,353 creating build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:01,354 copying lm_eval/tasks/include/few_shot_en/Malayalam/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:01,356 creating build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:52:01,357 copying lm_eval/tasks/include/few_shot_en/Kazakh/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:52:01,360 creating build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:01,361 copying lm_eval/tasks/include/few_shot_en/Uzbek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:01,364 creating build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:01,365 copying lm_eval/tasks/include/few_shot_en/Armenian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:01,367 creating build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:01,368 copying lm_eval/tasks/include/few_shot_en/Finnish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:01,371 creating build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:52:01,372 copying lm_eval/tasks/include/few_shot_en/Tagalog/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:52:01,374 creating build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:01,376 copying lm_eval/tasks/include/few_shot_en/Lithuanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:01,378 creating build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:01,379 copying lm_eval/tasks/include/few_shot_en/Hindi/utils.py -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:01,381 creating build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:01,383 copying lm_eval/tasks/include/few_shot_og/Indonesian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:01,385 creating build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:01,386 copying lm_eval/tasks/include/few_shot_og/Bulgarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:01,389 creating build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:01,390 copying lm_eval/tasks/include/few_shot_og/Bengali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:01,392 creating build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:01,393 copying lm_eval/tasks/include/few_shot_og/Turkish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:01,395 creating build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:01,396 copying lm_eval/tasks/include/few_shot_og/Belarusian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:01,399 creating build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:01,400 copying lm_eval/tasks/include/few_shot_og/Greek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:01,402 creating build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:01,403 copying lm_eval/tasks/include/few_shot_og/Hungarian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:01,405 creating build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:01,406 copying lm_eval/tasks/include/few_shot_og/Dutch/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:01,409 creating build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:52:01,410 copying lm_eval/tasks/include/few_shot_og/Korean/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:52:01,412 creating build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:01,413 copying lm_eval/tasks/include/few_shot_og/German/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:01,415 creating build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:01,416 copying lm_eval/tasks/include/few_shot_og/Polish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:01,418 creating build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:01,419 copying lm_eval/tasks/include/few_shot_og/Urdu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:01,422 creating build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:01,423 copying lm_eval/tasks/include/few_shot_og/North Macedonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:01,425 creating build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:01,426 copying lm_eval/tasks/include/few_shot_og/Ukrainian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:01,429 creating build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:01,430 copying lm_eval/tasks/include/few_shot_og/Chinese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:01,432 creating build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:01,433 copying lm_eval/tasks/include/few_shot_og/Portuguese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:01,435 creating build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:52:01,436 copying lm_eval/tasks/include/few_shot_og/Georgian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:52:01,439 creating build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:01,440 copying lm_eval/tasks/include/few_shot_og/French/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:01,443 creating build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:01,444 copying lm_eval/tasks/include/few_shot_og/Spanish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:01,446 creating build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:01,447 copying lm_eval/tasks/include/few_shot_og/Serbian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:01,450 creating build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:01,451 copying lm_eval/tasks/include/few_shot_og/Malay/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:01,453 creating build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:01,454 copying lm_eval/tasks/include/few_shot_og/Arabic/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:01,457 creating build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:01,458 copying lm_eval/tasks/include/few_shot_og/Vietnamese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:01,460 creating build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:01,461 copying lm_eval/tasks/include/few_shot_og/Russian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:01,464 creating build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:01,465 copying lm_eval/tasks/include/few_shot_og/Japanese/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:01,467 creating build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:01,468 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:01,470 creating build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:01,471 copying lm_eval/tasks/include/few_shot_og/Persian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:01,474 creating build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:01,475 copying lm_eval/tasks/include/few_shot_og/Albanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:01,477 creating build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:01,478 copying lm_eval/tasks/include/few_shot_og/Croatian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:01,481 creating build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:52:01,482 copying lm_eval/tasks/include/few_shot_og/Nepali/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:52:01,484 creating build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:01,485 copying lm_eval/tasks/include/few_shot_og/Italian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:01,487 creating build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:52:01,488 copying lm_eval/tasks/include/few_shot_og/Tamil/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:52:01,491 creating build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:52:01,492 copying lm_eval/tasks/include/few_shot_og/Basque/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:52:01,495 creating build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:52:01,496 copying lm_eval/tasks/include/few_shot_og/Hebrew/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:52:01,499 creating build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:01,500 copying lm_eval/tasks/include/few_shot_og/Estonian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:01,503 creating build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:01,504 copying lm_eval/tasks/include/few_shot_og/Telugu/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:01,506 creating build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:01,507 copying lm_eval/tasks/include/few_shot_og/Malayalam/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:01,510 creating build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:52:01,511 copying lm_eval/tasks/include/few_shot_og/Kazakh/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:52:01,513 creating build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:01,514 copying lm_eval/tasks/include/few_shot_og/Uzbek/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:01,516 creating build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:01,517 copying lm_eval/tasks/include/few_shot_og/Armenian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:01,520 creating build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:01,521 copying lm_eval/tasks/include/few_shot_og/Finnish/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:01,523 creating build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:52:01,524 copying lm_eval/tasks/include/few_shot_og/Tagalog/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:52:01,527 creating build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:01,528 copying lm_eval/tasks/include/few_shot_og/Lithuanian/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:01,530 creating build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:01,531 copying lm_eval/tasks/include/few_shot_og/Hindi/utils.py -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:01,533 creating build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:01,535 copying lm_eval/tasks/include/default/Indonesian/utils.py -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:01,537 creating build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:01,538 copying lm_eval/tasks/include/default/Bulgarian/utils.py -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:01,540 creating build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:01,541 copying lm_eval/tasks/include/default/Bengali/utils.py -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:01,544 creating build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:01,545 copying lm_eval/tasks/include/default/Turkish/utils.py -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:01,547 creating build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:01,548 copying lm_eval/tasks/include/default/Belarusian/utils.py -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:01,550 creating build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:01,551 copying lm_eval/tasks/include/default/Greek/utils.py -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:01,554 creating build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:01,555 copying lm_eval/tasks/include/default/Hungarian/utils.py -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:01,557 creating build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:01,558 copying lm_eval/tasks/include/default/Dutch/utils.py -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:01,561 creating build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T15:52:01,562 copying lm_eval/tasks/include/default/Korean/utils.py -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T15:52:01,564 creating build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:01,565 copying lm_eval/tasks/include/default/German/utils.py -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:01,568 creating build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:01,569 copying lm_eval/tasks/include/default/Polish/utils.py -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:01,571 creating build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:01,572 copying lm_eval/tasks/include/default/Urdu/utils.py -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:01,575 creating build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:01,576 copying lm_eval/tasks/include/default/North Macedonian/utils.py -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:01,578 creating build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:01,579 copying lm_eval/tasks/include/default/Ukrainian/utils.py -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:01,582 creating build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:01,583 copying lm_eval/tasks/include/default/Chinese/utils.py -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:01,585 creating build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:01,586 copying lm_eval/tasks/include/default/Portuguese/utils.py -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:01,589 creating build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T15:52:01,590 copying lm_eval/tasks/include/default/Georgian/utils.py -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T15:52:01,593 creating build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:01,594 copying lm_eval/tasks/include/default/French/utils.py -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:01,596 creating build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:01,597 copying lm_eval/tasks/include/default/Spanish/utils.py -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:01,600 creating build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:01,601 copying lm_eval/tasks/include/default/Serbian/utils.py -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:01,603 creating build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:01,604 copying lm_eval/tasks/include/default/Malay/utils.py -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:01,606 creating build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:01,607 copying lm_eval/tasks/include/default/Arabic/utils.py -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:01,610 creating build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:01,611 copying lm_eval/tasks/include/default/Vietnamese/utils.py -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:01,613 creating build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:01,614 copying lm_eval/tasks/include/default/Russian/utils.py -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:01,617 creating build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:01,618 copying lm_eval/tasks/include/default/Japanese/utils.py -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:01,620 creating build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:01,621 copying lm_eval/tasks/include/default/Azerbaijani/utils.py -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:01,623 creating build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:01,624 copying lm_eval/tasks/include/default/Persian/utils.py -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:01,627 creating build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:01,628 copying lm_eval/tasks/include/default/Albanian/utils.py -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:01,630 creating build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:01,631 copying lm_eval/tasks/include/default/Croatian/utils.py -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:01,633 creating build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T15:52:01,634 copying lm_eval/tasks/include/default/Nepali/utils.py -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T15:52:01,637 creating build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:01,638 copying lm_eval/tasks/include/default/Italian/utils.py -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:01,641 creating build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T15:52:01,642 copying lm_eval/tasks/include/default/Tamil/utils.py -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T15:52:01,644 creating build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T15:52:01,645 copying lm_eval/tasks/include/default/Basque/utils.py -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T15:52:01,647 creating build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:52:01,648 copying lm_eval/tasks/include/default/Hebrew/utils.py -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:52:01,650 creating build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:01,651 copying lm_eval/tasks/include/default/Estonian/utils.py -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:01,654 creating build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:01,655 copying lm_eval/tasks/include/default/Telugu/utils.py -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:01,657 creating build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:01,658 copying lm_eval/tasks/include/default/Malayalam/utils.py -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:01,660 creating build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T15:52:01,661 copying lm_eval/tasks/include/default/Kazakh/utils.py -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T15:52:01,664 creating build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:01,664 copying lm_eval/tasks/include/default/Uzbek/utils.py -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:01,667 creating build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:01,668 copying lm_eval/tasks/include/default/Armenian/utils.py -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:01,670 creating build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:01,671 copying lm_eval/tasks/include/default/Finnish/utils.py -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:01,673 creating build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:52:01,674 copying lm_eval/tasks/include/default/Tagalog/utils.py -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:52:01,676 creating build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:01,677 copying lm_eval/tasks/include/default/Lithuanian/utils.py -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:01,679 creating build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:01,680 copying lm_eval/tasks/include/default/Hindi/utils.py -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:01,683 creating build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:01,683 copying lm_eval/tasks/mmlusr/question_only/utils.py -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:01,686 creating build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:01,687 copying lm_eval/tasks/mmlusr/answer_only/utils.py -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:01,690 creating build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:01,691 copying lm_eval/tasks/mmlusr/question_and_answer/utils.py -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:01,694 creating build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:01,695 copying lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:01,698 copying lm_eval/tasks/code_x_glue/code-text/utils.py -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:01,700 creating build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:01,701 copying lm_eval/tasks/galician_bench/flores_gl/create_yamls_flores_gl.py -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:01,704 creating build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:52:01,705 copying lm_eval/tasks/eq_bench/multilingual/utils.py -> build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:52:01,708 creating build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:01,710 copying lm_eval/tasks/portuguese_bench/flores_pt/create_yamls_flores_pt.py -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:01,718 creating build/lib/lm_eval/tasks/glue/mnli 2026-05-11T15:52:01,720 copying lm_eval/tasks/glue/mnli/utils.py -> build/lib/lm_eval/tasks/glue/mnli 2026-05-11T15:52:01,722 creating build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:01,724 copying lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:01,726 copying lm_eval/tasks/gpqa/cot_n_shot/utils.py -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:01,729 creating build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:01,730 copying lm_eval/tasks/gpqa/n_shot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:01,732 copying lm_eval/tasks/gpqa/n_shot/utils.py -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:01,734 creating build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:01,735 copying lm_eval/tasks/gpqa/generative/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:01,737 copying lm_eval/tasks/gpqa/generative/utils.py -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:01,739 creating build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:01,740 copying lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:01,742 copying lm_eval/tasks/gpqa/cot_zeroshot/utils.py -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:01,745 creating build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:01,746 copying lm_eval/tasks/gpqa/zeroshot/_generate_configs.py -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:01,748 copying lm_eval/tasks/gpqa/zeroshot/utils.py -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:01,750 creating build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T15:52:01,752 copying lm_eval/tasks/afrobench/naijarc/utils.py -> build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T15:52:01,754 creating build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T15:52:01,755 copying lm_eval/tasks/afrobench/sib/utils.py -> build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T15:52:01,758 creating build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:52:01,759 copying lm_eval/tasks/afrobench/masakhanews/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:52:01,762 creating build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T15:52:01,763 copying lm_eval/tasks/afrobench/belebele/utils.py -> build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T15:52:01,766 creating build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:52:01,767 copying lm_eval/tasks/afrobench/masakhapos/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:52:01,770 copying lm_eval/tasks/afrobench/masakhapos/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:52:01,772 creating build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T15:52:01,773 copying lm_eval/tasks/afrobench/salt/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T15:52:01,777 creating build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T15:52:01,778 copying lm_eval/tasks/afrobench/ntrex/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T15:52:01,781 creating build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T15:52:01,782 copying lm_eval/tasks/afrobench/flores/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T15:52:01,786 creating build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T15:52:01,787 copying lm_eval/tasks/afrobench/injongointent/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T15:52:01,791 creating build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:52:01,792 copying lm_eval/tasks/afrobench/afrisenti/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:52:01,795 creating build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T15:52:01,797 copying lm_eval/tasks/afrobench/adr/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T15:52:01,800 creating build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T15:52:01,801 copying lm_eval/tasks/afrobench/xlsum/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T15:52:01,804 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:52:01,805 copying lm_eval/tasks/afrobench/uhura-arc-easy/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:52:01,808 creating build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:52:01,810 copying lm_eval/tasks/afrobench/masakhaner/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:52:01,813 creating build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T15:52:01,815 copying lm_eval/tasks/afrobench/afriqa/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T15:52:01,818 creating build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T15:52:01,820 copying lm_eval/tasks/afrobench/mafand/gen_utils.py -> build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T15:52:01,823 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:52:01,824 copying lm_eval/tasks/afrobench/openai_mmlu/utils.py -> build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:52:01,829 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:01,830 copying lm_eval/tasks/afrobench/sib/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:01,833 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:01,835 copying lm_eval/tasks/afrobench/sib/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:01,838 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:01,839 copying lm_eval/tasks/afrobench/sib/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:01,842 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:01,843 copying lm_eval/tasks/afrobench/sib/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:01,846 creating build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:01,847 copying lm_eval/tasks/afrobench/sib/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:01,850 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:01,851 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:01,854 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:01,855 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:01,858 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:01,859 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:01,862 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:01,863 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:01,866 creating build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:01,867 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:01,872 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:01,874 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:01,877 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:01,878 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:01,881 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:01,882 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:01,885 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:01,886 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:01,889 creating build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:01,891 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:01,902 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:01,904 copying lm_eval/tasks/afrobench/injongointent/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:01,907 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:01,908 copying lm_eval/tasks/afrobench/injongointent/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:01,911 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:01,912 copying lm_eval/tasks/afrobench/injongointent/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:01,915 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:01,916 copying lm_eval/tasks/afrobench/injongointent/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:01,919 creating build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:01,920 copying lm_eval/tasks/afrobench/injongointent/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:01,923 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:01,925 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:01,927 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:01,929 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:01,931 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:01,932 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:01,935 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:01,936 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:01,939 creating build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:01,940 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:01,943 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:01,945 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:01,947 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:01,950 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:01,951 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:01,953 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:01,956 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:01,957 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:01,959 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:01,962 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:01,964 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:01,966 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:01,969 creating build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:01,970 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/xx.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:01,972 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:01,977 creating build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:01,979 copying lm_eval/tasks/afrobench/xlsum/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:01,981 creating build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:01,983 copying lm_eval/tasks/afrobench/xlsum/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:01,985 creating build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:01,986 copying lm_eval/tasks/afrobench/xlsum/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:01,989 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:01,990 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:01,993 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:01,994 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:01,997 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:01,998 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:02,000 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:02,002 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:02,004 creating build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:02,005 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:02,008 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:02,010 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:02,013 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:02,014 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:02,017 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:02,019 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:02,022 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:02,023 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:02,026 creating build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:02,028 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:02,031 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:02,032 copying lm_eval/tasks/afrobench/afriqa/prompt_2/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:02,035 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:02,036 copying lm_eval/tasks/afrobench/afriqa/prompt_1/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:02,039 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:02,041 copying lm_eval/tasks/afrobench/afriqa/prompt_5/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:02,043 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:02,045 copying lm_eval/tasks/afrobench/afriqa/prompt_3/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:02,047 creating build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:02,049 copying lm_eval/tasks/afrobench/afriqa/prompt_4/utils.py -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:02,053 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:02,055 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:02,058 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:02,059 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:02,062 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:02,064 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:02,067 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:02,069 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:02,072 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:02,073 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:02,076 creating build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:02,078 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/utils.py -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:02,085 creating build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:02,086 copying lm_eval/tasks/openai-mmmlu/default/utils.py -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:02,090 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:02,092 copying lm_eval/tasks/llama3/instruct/mmlu_hi/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:02,096 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:02,097 copying lm_eval/tasks/llama3/instruct/mmlu_fr/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:02,100 creating build/lib/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T15:52:02,101 copying lm_eval/tasks/llama3/instruct/arc_challenge/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T15:52:02,104 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:02,106 copying lm_eval/tasks/llama3/instruct/mmlu_it/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:02,109 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:02,110 copying lm_eval/tasks/llama3/instruct/mmlu_th/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:02,114 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:02,116 copying lm_eval/tasks/llama3/instruct/mmlu_de/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:02,119 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:02,121 copying lm_eval/tasks/llama3/instruct/mmlu_es/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:02,124 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:02,125 copying lm_eval/tasks/llama3/instruct/mmlu_pro/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:02,128 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:02,130 copying lm_eval/tasks/llama3/instruct/mmlu_pt/utils.py -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:02,133 creating build/lib/lm_eval/api/task 2026-05-11T15:52:02,134 copying lm_eval/api/task/_ll.py -> build/lib/lm_eval/api/task 2026-05-11T15:52:02,137 copying lm_eval/api/task/__init__.py -> build/lib/lm_eval/api/task 2026-05-11T15:52:02,139 copying lm_eval/api/task/_task.py -> build/lib/lm_eval/api/task 2026-05-11T15:52:02,143 copying lm_eval/api/task/_generation.py -> build/lib/lm_eval/api/task 2026-05-11T15:52:02,146 creating build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,147 copying lm_eval/api/metrics/_types.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,149 copying lm_eval/api/metrics/ll.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,152 copying lm_eval/api/metrics/__init__.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,154 copying lm_eval/api/metrics/corpus.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,156 copying lm_eval/api/metrics/generation.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,159 copying lm_eval/api/metrics/metric.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,161 copying lm_eval/api/metrics/stderr.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,164 copying lm_eval/api/metrics/reduce.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,166 copying lm_eval/api/metrics/aggregations.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,168 copying lm_eval/api/metrics/utils.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,171 copying lm_eval/api/metrics/results.py -> build/lib/lm_eval/api/metrics 2026-05-11T15:52:02,173 creating build/lib/lm_eval/config/formats 2026-05-11T15:52:02,175 copying lm_eval/config/formats/__init__.py -> build/lib/lm_eval/config/formats 2026-05-11T15:52:02,177 copying lm_eval/config/formats/_base.py -> build/lib/lm_eval/config/formats 2026-05-11T15:52:02,180 copying lm_eval/config/formats/_multiple_choice.py -> build/lib/lm_eval/config/formats 2026-05-11T15:52:02,182 copying lm_eval/config/formats/_generation.py -> build/lib/lm_eval/config/formats 2026-05-11T15:52:02,192 running egg_info 2026-05-11T15:52:02,202 writing lm_eval.egg-info/PKG-INFO 2026-05-11T15:52:02,224 writing dependency_links to lm_eval.egg-info/dependency_links.txt 2026-05-11T15:52:02,226 writing entry points to lm_eval.egg-info/entry_points.txt 2026-05-11T15:52:02,236 writing requirements to lm_eval.egg-info/requires.txt 2026-05-11T15:52:02,239 writing top-level names to lm_eval.egg-info/top_level.txt 2026-05-11T15:52:05,126 reading manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:52:06,139 reading manifest template 'MANIFEST.in' 2026-05-11T15:52:06,139 warning: manifest_maker: MANIFEST.in, line 1: 'recursive-include' expects ... 2026-05-11T15:52:06,141 adding license file 'LICENSE.md' 2026-05-11T15:52:08,288 writing manifest file 'lm_eval.egg-info/SOURCES.txt' 2026-05-11T15:52:11,805 copying lm_eval/tasks/noticia/noticia.yaml -> build/lib/lm_eval/tasks/noticia 2026-05-11T15:52:11,808 copying lm_eval/tasks/arab_culture/arab_culture_algeria.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,810 copying lm_eval/tasks/arab_culture/arab_culture_ksa.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,813 copying lm_eval/tasks/arab_culture/arab_culture_tunisia.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,816 copying lm_eval/tasks/arab_culture/_arab_culture.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,818 copying lm_eval/tasks/arab_culture/arab_culture_uae.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,821 copying lm_eval/tasks/arab_culture/arab_culture_libya.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,824 copying lm_eval/tasks/arab_culture/arab_culture_egypt.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,826 copying lm_eval/tasks/arab_culture/arab_culture_syria.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,829 copying lm_eval/tasks/arab_culture/arab_culture_lebanon.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,831 copying lm_eval/tasks/arab_culture/arab_culture_palestine.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,834 copying lm_eval/tasks/arab_culture/_arab_culture_gulf.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,836 copying lm_eval/tasks/arab_culture/_arab_culture_north_africa.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,839 copying lm_eval/tasks/arab_culture/arab_culture_sudan.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,842 copying lm_eval/tasks/arab_culture/arab_culture_morocco.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,845 copying lm_eval/tasks/arab_culture/_arab_culture_nile_valley.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,847 copying lm_eval/tasks/arab_culture/arab_culture_jordan.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,850 copying lm_eval/tasks/arab_culture/_arab_culture_levant.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,852 copying lm_eval/tasks/arab_culture/arab_culture_yemen.yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:11,855 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,857 copying lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,860 copying lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,863 copying lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,865 copying lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,868 copying lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,870 copying lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,873 copying lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,876 copying lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,878 copying lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,881 copying lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,884 copying lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,886 copying lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,889 copying lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,892 copying lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,894 copying lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,897 copying lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,900 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,902 copying lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,905 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,908 copying lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,910 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,913 copying lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,916 copying lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,918 copying lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,921 copying lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,923 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,926 copying lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,929 copying lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,931 copying lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,934 copying lm_eval/tasks/tmmluplus/default/_tmmluplus.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,936 copying lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,939 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_STEM.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,941 copying lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,944 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_social_sciences.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,947 copying lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,949 copying lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,952 copying lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,955 copying lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,957 copying lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,960 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,963 copying lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,965 copying lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,968 copying lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,971 copying lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,973 copying lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,976 copying lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,979 copying lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,982 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,984 copying lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,987 copying lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,990 copying lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,993 copying lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,996 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_humanities.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:11,998 copying lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,001 copying lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,003 copying lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,006 copying lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,009 copying lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,011 copying lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,014 copying lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,017 copying lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,019 copying lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,022 copying lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,025 copying lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,027 copying lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,030 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_other.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,033 copying lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,035 copying lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,038 copying lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,041 copying lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,043 copying lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:12,046 copying lm_eval/tasks/mimic_repsum/mimic_repsum.yaml -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T15:52:12,048 copying lm_eval/tasks/mimic_repsum/mimic_repsum_perplexity.yaml -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T15:52:12,051 copying lm_eval/tasks/cnn_dailymail/cnn_dailymail.yaml -> build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T15:52:12,054 copying lm_eval/tasks/aradice/aradice.yaml -> build/lib/lm_eval/tasks/aradice 2026-05-11T15:52:12,057 copying lm_eval/tasks/aradice/winogrande/winogrande_msa.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:12,060 copying lm_eval/tasks/aradice/winogrande/winogrande_eng.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:12,062 copying lm_eval/tasks/aradice/winogrande/winogrande_egy.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:12,065 copying lm_eval/tasks/aradice/winogrande/winogrande_lev.yaml -> build/lib/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:12,068 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,070 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,073 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,076 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,079 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,082 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,084 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,087 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,089 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,092 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,094 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,097 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,100 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,105 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,107 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,110 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,112 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,114 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,117 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,119 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,121 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,123 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,126 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,128 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,130 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,132 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,135 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,137 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,139 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,141 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,143 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,146 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,148 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,150 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,152 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,155 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,157 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,159 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,162 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,164 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,166 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:12,168 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,171 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,173 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,175 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,178 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,180 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,182 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,185 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,187 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,189 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,191 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,194 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,196 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,198 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,200 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,202 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,205 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,207 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,209 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,212 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,214 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,216 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,218 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,221 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,223 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,225 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,227 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,230 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,232 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,234 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,237 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,239 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,242 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,244 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,246 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,249 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,251 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,253 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,255 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,257 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,260 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:12,262 copying lm_eval/tasks/aradice/openbookqa/openbookqa_msa.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:12,264 copying lm_eval/tasks/aradice/openbookqa/openbookqa_eng.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:12,267 copying lm_eval/tasks/aradice/openbookqa/openbookqa_lev.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:12,269 copying lm_eval/tasks/aradice/openbookqa/openbookqa_egy.yaml -> build/lib/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:12,272 copying lm_eval/tasks/aradice/cultural-benchmark/qatar.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:12,274 copying lm_eval/tasks/aradice/cultural-benchmark/palestine.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:12,276 copying lm_eval/tasks/aradice/cultural-benchmark/egypt.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:12,279 copying lm_eval/tasks/aradice/cultural-benchmark/lebanon.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:12,281 copying lm_eval/tasks/aradice/cultural-benchmark/syria.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:12,284 copying lm_eval/tasks/aradice/cultural-benchmark/jordan.yaml -> build/lib/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:12,286 copying lm_eval/tasks/aradice/boolq/MSA/boolq_msa.yaml -> build/lib/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:12,288 copying lm_eval/tasks/aradice/boolq/LEV/boolq_lev.yaml -> build/lib/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:12,291 copying lm_eval/tasks/aradice/boolq/EGY/boolq_egy.yaml -> build/lib/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:12,293 copying lm_eval/tasks/aradice/boolq/ENG/boolq_eng.yaml -> build/lib/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:12,296 copying lm_eval/tasks/aradice/piqa/piqa_lev.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:12,298 copying lm_eval/tasks/aradice/piqa/piqa_msa.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:12,300 copying lm_eval/tasks/aradice/piqa/piqa_egy.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:12,303 copying lm_eval/tasks/aradice/piqa/piqa_eng.yaml -> build/lib/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:12,305 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_eng.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:12,308 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_msa.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:12,310 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_egy.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:12,313 copying lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_lev.yaml -> build/lib/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:12,315 creating build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,317 copying lm_eval/tasks/mmlu-redux/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,319 copying lm_eval/tasks/mmlu-redux/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,321 copying lm_eval/tasks/mmlu-redux/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,324 copying lm_eval/tasks/mmlu-redux/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,326 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,328 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,330 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,333 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,335 copying lm_eval/tasks/mmlu-redux/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,337 copying lm_eval/tasks/mmlu-redux/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,340 copying lm_eval/tasks/mmlu-redux/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,342 copying lm_eval/tasks/mmlu-redux/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,345 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,347 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,349 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,351 copying lm_eval/tasks/mmlu-redux/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,354 copying lm_eval/tasks/mmlu-redux/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,356 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,358 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,361 copying lm_eval/tasks/mmlu-redux/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,363 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,366 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,368 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,370 copying lm_eval/tasks/mmlu-redux/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,373 copying lm_eval/tasks/mmlu-redux/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,375 copying lm_eval/tasks/mmlu-redux/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,377 copying lm_eval/tasks/mmlu-redux/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,380 copying lm_eval/tasks/mmlu-redux/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,382 copying lm_eval/tasks/mmlu-redux/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,384 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,386 copying lm_eval/tasks/mmlu-redux/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,388 copying lm_eval/tasks/mmlu-redux/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,390 copying lm_eval/tasks/mmlu-redux/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,392 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,395 copying lm_eval/tasks/mmlu-redux/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,397 copying lm_eval/tasks/mmlu-redux/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,399 copying lm_eval/tasks/mmlu-redux/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,401 copying lm_eval/tasks/mmlu-redux/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,404 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,406 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,408 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,411 copying lm_eval/tasks/mmlu-redux/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,413 copying lm_eval/tasks/mmlu-redux/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,415 copying lm_eval/tasks/mmlu-redux/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,418 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,420 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,422 copying lm_eval/tasks/mmlu-redux/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,425 copying lm_eval/tasks/mmlu-redux/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,427 copying lm_eval/tasks/mmlu-redux/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,429 copying lm_eval/tasks/mmlu-redux/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,431 copying lm_eval/tasks/mmlu-redux/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,434 copying lm_eval/tasks/mmlu-redux/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,436 copying lm_eval/tasks/mmlu-redux/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,439 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,441 copying lm_eval/tasks/mmlu-redux/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,443 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,445 copying lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,448 copying lm_eval/tasks/mmlu-redux/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:12,450 creating build/lib/lm_eval/tasks/mc_taco 2026-05-11T15:52:12,451 copying lm_eval/tasks/mc_taco/default.yaml -> build/lib/lm_eval/tasks/mc_taco 2026-05-11T15:52:12,453 copying lm_eval/tasks/humaneval/humaneval.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:52:12,455 copying lm_eval/tasks/humaneval/humaneval_64_instruct.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:52:12,457 copying lm_eval/tasks/humaneval/humaneval_64.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:52:12,460 copying lm_eval/tasks/humaneval/humaneval_plus.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:52:12,462 copying lm_eval/tasks/humaneval/humaneval_instruct.yaml -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:52:12,464 creating build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,465 copying lm_eval/tasks/multiblimp/multiblimp_por.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,467 copying lm_eval/tasks/multiblimp/multiblimp_kxh.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,469 copying lm_eval/tasks/multiblimp/multiblimp_wol.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,472 copying lm_eval/tasks/multiblimp/multiblimp_kpv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,474 copying lm_eval/tasks/multiblimp/multiblimp_uzb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,476 copying lm_eval/tasks/multiblimp/multiblimp_eng.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,478 copying lm_eval/tasks/multiblimp/multiblimp_quc.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,481 copying lm_eval/tasks/multiblimp/multiblimp_hye.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,483 copying lm_eval/tasks/multiblimp/multiblimp_got.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,485 copying lm_eval/tasks/multiblimp/multiblimp_slv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,487 copying lm_eval/tasks/multiblimp/multiblimp_orv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,489 copying lm_eval/tasks/multiblimp/multiblimp_sms.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,492 copying lm_eval/tasks/multiblimp/multiblimp_eus.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,494 copying lm_eval/tasks/multiblimp/multiblimp_rus.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,496 copying lm_eval/tasks/multiblimp/multiblimp_vep.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,498 copying lm_eval/tasks/multiblimp/multiblimp_nds.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,500 copying lm_eval/tasks/multiblimp/multiblimp_amh.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,502 copying lm_eval/tasks/multiblimp/multiblimp_tur.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,505 copying lm_eval/tasks/multiblimp/multiblimp_lat.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,507 copying lm_eval/tasks/multiblimp/multiblimp_urd.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,509 copying lm_eval/tasks/multiblimp/multiblimp_glg.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,511 copying lm_eval/tasks/multiblimp/multiblimp_ota.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,513 copying lm_eval/tasks/multiblimp/multiblimp_krl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,515 copying lm_eval/tasks/multiblimp/multiblimp_abk.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,518 copying lm_eval/tasks/multiblimp/multiblimp_fin.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,520 copying lm_eval/tasks/multiblimp/multiblimp_urb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,522 copying lm_eval/tasks/multiblimp/multiblimp_hun.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,524 copying lm_eval/tasks/multiblimp/multiblimp_xpg.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,526 copying lm_eval/tasks/multiblimp/multiblimp_mdf.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,528 copying lm_eval/tasks/multiblimp/multiblimp_wbp.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,530 copying lm_eval/tasks/multiblimp/multiblimp_yrl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,532 copying lm_eval/tasks/multiblimp/multiblimp_nhi.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,534 copying lm_eval/tasks/multiblimp/multiblimp_pcm.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,536 copying lm_eval/tasks/multiblimp/multiblimp_lav.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,539 copying lm_eval/tasks/multiblimp/multiblimp_frm.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,541 copying lm_eval/tasks/multiblimp/multiblimp_olo.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,543 copying lm_eval/tasks/multiblimp/multiblimp_kmr.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,545 copying lm_eval/tasks/multiblimp/multiblimp_ell.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,547 copying lm_eval/tasks/multiblimp/multiblimp_cym.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,550 copying lm_eval/tasks/multiblimp/multiblimp_lit.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,552 copying lm_eval/tasks/multiblimp/multiblimp_dan.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,554 copying lm_eval/tasks/multiblimp/multiblimp_bel.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,556 copying lm_eval/tasks/multiblimp/multiblimp_kat.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,558 copying lm_eval/tasks/multiblimp/multiblimp_ttc.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,561 copying lm_eval/tasks/multiblimp/multiblimp_fas.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,563 copying lm_eval/tasks/multiblimp/multiblimp_hbs.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,565 copying lm_eval/tasks/multiblimp/multiblimp_hit.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,567 copying lm_eval/tasks/multiblimp/multiblimp_nld.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,570 copying lm_eval/tasks/multiblimp/multiblimp_aqz.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,572 copying lm_eval/tasks/multiblimp/multiblimp_swe.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,574 copying lm_eval/tasks/multiblimp/multiblimp_fao.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,576 copying lm_eval/tasks/multiblimp/multiblimp_pol.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,579 copying lm_eval/tasks/multiblimp/multiblimp_heb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,581 copying lm_eval/tasks/multiblimp/multiblimp_ron.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,583 copying lm_eval/tasks/multiblimp/multiblimp_xnr.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,585 copying lm_eval/tasks/multiblimp/multiblimp_deu.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,587 copying lm_eval/tasks/multiblimp/multiblimp_bor.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,589 copying lm_eval/tasks/multiblimp/multiblimp_tpn.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,591 copying lm_eval/tasks/multiblimp/multiblimp_hin.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,594 copying lm_eval/tasks/multiblimp/multiblimp_san.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,596 copying lm_eval/tasks/multiblimp/multiblimp_xcl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,598 copying lm_eval/tasks/multiblimp/multiblimp_bul.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,600 copying lm_eval/tasks/multiblimp/multiblimp_spa.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,602 copying lm_eval/tasks/multiblimp/multiblimp_bre.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,604 copying lm_eval/tasks/multiblimp/multiblimp_sme.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,607 copying lm_eval/tasks/multiblimp/multiblimp_egy.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,609 copying lm_eval/tasks/multiblimp/multiblimp_bho.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,611 copying lm_eval/tasks/multiblimp/multiblimp_koi.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,613 copying lm_eval/tasks/multiblimp/multiblimp_kaz.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,615 copying lm_eval/tasks/multiblimp/multiblimp_guj.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,618 copying lm_eval/tasks/multiblimp/multiblimp_apu.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,620 copying lm_eval/tasks/multiblimp/multiblimp_kir.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,622 copying lm_eval/tasks/multiblimp/multiblimp_hbo.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,624 copying lm_eval/tasks/multiblimp/multiblimp_mar.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,626 copying lm_eval/tasks/multiblimp/multiblimp_ces.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,629 copying lm_eval/tasks/multiblimp/multiblimp_ben.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,631 copying lm_eval/tasks/multiblimp/multiblimp_mkd.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,633 copying lm_eval/tasks/multiblimp/multiblimp_est.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,635 copying lm_eval/tasks/multiblimp/multiblimp_tam.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,638 copying lm_eval/tasks/multiblimp/multiblimp_lij.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,640 copying lm_eval/tasks/multiblimp/multiblimp_chu.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,642 copying lm_eval/tasks/multiblimp/multiblimp_bua.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,644 copying lm_eval/tasks/multiblimp/multiblimp_ita.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,647 copying lm_eval/tasks/multiblimp/multiblimp_uig.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,649 copying lm_eval/tasks/multiblimp/multiblimp_sqi.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,651 copying lm_eval/tasks/multiblimp/multiblimp_myv.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,654 copying lm_eval/tasks/multiblimp/multiblimp_gle.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,656 copying lm_eval/tasks/multiblimp/multiblimp_isl.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,658 copying lm_eval/tasks/multiblimp/multiblimp_sah.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,660 copying lm_eval/tasks/multiblimp/multiblimp_azz.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,663 copying lm_eval/tasks/multiblimp/multiblimp_hyw.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,665 copying lm_eval/tasks/multiblimp/multiblimp_arb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,667 copying lm_eval/tasks/multiblimp/multiblimp_grc.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,669 copying lm_eval/tasks/multiblimp/multiblimp_hsb.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,671 copying lm_eval/tasks/multiblimp/multiblimp_fro.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,674 copying lm_eval/tasks/multiblimp/multiblimp_ukr.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,676 copying lm_eval/tasks/multiblimp/multiblimp_slk.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,678 copying lm_eval/tasks/multiblimp/multiblimp_cat.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,680 copying lm_eval/tasks/multiblimp/multiblimp_aln.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,682 copying lm_eval/tasks/multiblimp/multiblimp_fra.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,685 copying lm_eval/tasks/multiblimp/multiblimp_gla.yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:12,687 creating build/lib/lm_eval/tasks/commonsense_qa 2026-05-11T15:52:12,688 copying lm_eval/tasks/commonsense_qa/default.yaml -> build/lib/lm_eval/tasks/commonsense_qa 2026-05-11T15:52:12,690 creating build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:12,691 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Material.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:12,693 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Biology.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:12,696 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Chemistry.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:12,698 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/_sciknoweval_mcqa.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:12,700 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Physics.yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:12,702 copying lm_eval/tasks/e2lmc/noor/noor_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,705 copying lm_eval/tasks/e2lmc/noor/noor_college_mathematics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,707 copying lm_eval/tasks/e2lmc/noor/noor_college_physics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,709 copying lm_eval/tasks/e2lmc/noor/noor_high_school_statistics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,711 copying lm_eval/tasks/e2lmc/noor/noor_electrical_engineering.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,714 copying lm_eval/tasks/e2lmc/noor/noor_machine_learning.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,716 copying lm_eval/tasks/e2lmc/noor/noor_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,718 copying lm_eval/tasks/e2lmc/noor/noor_college_computer_science.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,721 copying lm_eval/tasks/e2lmc/noor/_noor.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,723 copying lm_eval/tasks/e2lmc/noor/_noor_stem.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,725 copying lm_eval/tasks/e2lmc/noor/noor_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,727 copying lm_eval/tasks/e2lmc/noor/noor_abstract_algebra.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,730 copying lm_eval/tasks/e2lmc/noor/noor_conceptual_physics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,732 copying lm_eval/tasks/e2lmc/noor/noor_high_school_physics.yaml -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:12,734 copying lm_eval/tasks/e2lmc/mmlu_early_training/mmlu_early_training.yaml -> build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:12,737 copying lm_eval/tasks/paloma/paloma_wikitext_103.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,739 copying lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,741 copying lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,743 copying lm_eval/tasks/paloma/paloma_redpajama.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,745 copying lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,748 copying lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,750 copying lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,752 copying lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,754 copying lm_eval/tasks/paloma/paloma_gab.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,756 copying lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,759 copying lm_eval/tasks/paloma/paloma_c4_100_domains.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,761 copying lm_eval/tasks/paloma/paloma_mc4.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,763 copying lm_eval/tasks/paloma/paloma_c4_en.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,765 copying lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,767 copying lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,770 copying lm_eval/tasks/paloma/paloma_ptb.yaml -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:12,772 copying lm_eval/tasks/c4/c4.yaml -> build/lib/lm_eval/tasks/c4 2026-05-11T15:52:12,774 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,776 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,779 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,781 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,783 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,786 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,788 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,790 copying lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_lite_mr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,793 copying lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_mr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,795 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,797 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,799 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,801 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,804 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,806 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,808 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,810 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,812 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,814 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,817 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,819 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,821 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,823 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,825 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,827 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,830 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,832 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,834 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,836 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,838 copying lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:12,841 copying lm_eval/tasks/mmlu_prox/de/_mmlu_prox_de.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,843 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,845 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,848 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,850 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,852 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,854 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,857 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,859 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,861 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,864 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,866 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,868 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,871 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,873 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,875 copying lm_eval/tasks/mmlu_prox/de/_mmlu_prox_lite_de.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,877 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,880 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,882 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,884 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,886 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,889 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,891 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,893 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,895 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,897 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,899 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,902 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,904 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,906 copying lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:12,908 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,910 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,913 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,915 copying lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_lite_yo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,917 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,920 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,922 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,924 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,926 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,929 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,931 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,933 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,936 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,938 copying lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_yo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,940 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,943 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,945 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,947 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,950 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,952 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,954 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,956 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,958 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,960 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,963 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,965 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,967 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,969 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,971 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,974 copying lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:12,976 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,978 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,980 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,983 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,985 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,987 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,989 copying lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_ru.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,992 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,994 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,996 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:12,999 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,001 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,004 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,006 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,008 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,010 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,013 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,015 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,017 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,020 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,022 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,024 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,026 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,029 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,031 copying lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_lite_ru.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,033 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,035 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,038 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,040 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,042 copying lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:13,044 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,046 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,049 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,051 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,053 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,056 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,058 copying lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_zh.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,060 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,062 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,065 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,067 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,070 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,072 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,074 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,076 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,079 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,081 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,084 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,086 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,088 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,091 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,093 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,095 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,098 copying lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_lite_zh.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,100 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,102 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,104 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,106 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,108 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,111 copying lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:13,113 copying lm_eval/tasks/mmlu_prox/af/_mmlu_prox_lite_af.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,115 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,117 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,119 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,122 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,124 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,126 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,129 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,131 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,133 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,135 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,138 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,140 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,142 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,145 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,147 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,149 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,152 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,154 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,156 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,159 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,161 copying lm_eval/tasks/mmlu_prox/af/_mmlu_prox_af.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,163 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,166 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,168 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,170 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,172 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,175 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,177 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,179 copying lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:13,182 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,184 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,187 copying lm_eval/tasks/mmlu_prox/th/_mmlu_prox_lite_th.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,190 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,192 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,194 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,196 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,199 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,201 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,204 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,206 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,208 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,211 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,213 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,215 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,217 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,220 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,223 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,225 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,228 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,231 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,233 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,236 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,238 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,240 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,243 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,245 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,247 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,249 copying lm_eval/tasks/mmlu_prox/th/_mmlu_prox_th.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,251 copying lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:13,254 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,256 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,258 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,260 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,263 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,265 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,268 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,271 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,273 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,276 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,279 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,281 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,283 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,286 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,288 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,290 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,293 copying lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_lite_zu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,295 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,297 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,299 copying lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_zu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,302 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,304 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,306 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,308 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,311 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,313 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,315 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,317 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,320 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,322 copying lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:13,324 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,326 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,328 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,330 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,332 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,334 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,336 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,339 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,341 copying lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_lite_ne.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,343 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,345 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,347 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,350 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,352 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,354 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,356 copying lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_ne.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,358 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,360 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,363 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,365 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,367 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,369 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,372 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,374 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,376 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,378 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,381 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,383 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,385 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,387 copying lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:13,390 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,392 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,394 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,396 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,399 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,401 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,403 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,405 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,407 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,409 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,412 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,414 copying lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_lite_vi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,416 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,418 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,421 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,423 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,425 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,428 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,430 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,432 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,434 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,436 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,439 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,441 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,443 copying lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_vi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,446 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,448 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,450 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,453 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,455 copying lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:13,458 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,460 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,462 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,464 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,466 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,469 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,471 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,473 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,475 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,477 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,479 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,482 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,484 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,486 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,489 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,491 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,493 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,496 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,498 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,500 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,502 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,505 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,507 copying lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_lite_ko.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,509 copying lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_ko.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,512 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,514 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,516 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,518 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,521 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,523 copying lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:13,525 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,527 copying lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_lite_bn.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,530 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,532 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,534 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,536 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,538 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,541 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,543 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,545 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,548 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,550 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,552 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,554 copying lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_bn.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,557 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,559 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,561 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,564 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,566 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,568 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,571 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,573 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,575 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,577 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,580 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,582 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,584 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,587 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,589 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,592 copying lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:13,594 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,596 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,598 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,601 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,603 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,605 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,607 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,609 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,612 copying lm_eval/tasks/mmlu_prox/en/_mmlu_prox_en.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,614 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,616 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,619 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,621 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,623 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,626 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,628 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,630 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,633 copying lm_eval/tasks/mmlu_prox/en/_mmlu_prox_lite_en.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,635 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,637 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,640 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,642 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,644 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,646 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,649 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,651 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,653 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,656 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,658 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,660 copying lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:13,663 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,665 copying lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_uk.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,667 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,670 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,672 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,674 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,677 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,679 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,681 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,683 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,686 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,688 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,690 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,693 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,695 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,697 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,699 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,702 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,704 copying lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_lite_uk.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,707 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,709 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,712 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,714 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,716 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,719 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,721 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,723 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,726 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,728 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,731 copying lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:13,733 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,736 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,738 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,740 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,743 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,745 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,748 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,750 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,752 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,754 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,756 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,759 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,761 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,763 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,765 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,767 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,770 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,772 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,774 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,777 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,779 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,781 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,783 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,786 copying lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_lite_sw.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,788 copying lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_sw.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,790 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,793 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,795 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,797 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,799 copying lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:13,802 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,804 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,806 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,809 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,811 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,813 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,816 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,818 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,820 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,823 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,825 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,827 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,829 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,832 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,834 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,836 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,838 copying lm_eval/tasks/mmlu_prox/es/_mmlu_prox_lite_es.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,841 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,843 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,845 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,848 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,850 copying lm_eval/tasks/mmlu_prox/es/_mmlu_prox_es.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,852 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,855 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,857 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,859 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,862 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,864 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,867 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,869 copying lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:13,871 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,874 copying lm_eval/tasks/mmlu_prox/te/_mmlu_prox_te.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,876 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,878 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,881 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,883 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,885 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,888 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,890 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,893 copying lm_eval/tasks/mmlu_prox/te/_mmlu_prox_lite_te.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,895 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,897 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,900 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,902 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,904 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,907 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,909 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,911 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,914 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,916 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,918 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,920 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,922 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,925 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,927 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,929 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,931 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,934 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,936 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,938 copying lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:13,940 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,943 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,945 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,948 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,951 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,953 copying lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_lite_ja.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,956 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,958 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,961 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,964 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,967 copying lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_ja.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,969 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,972 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,975 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,977 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,980 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,982 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,985 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,988 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,990 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,993 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,996 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:13,998 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,001 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,003 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,006 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,008 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,011 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,014 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,016 copying lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:14,019 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,021 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,024 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,026 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,029 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,032 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,034 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,037 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,039 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,042 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,044 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,047 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,050 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,052 copying lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_lite_hi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,054 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,056 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,059 copying lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_hi.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,061 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,063 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,065 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,067 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,070 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,072 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,074 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,076 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,078 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,080 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,083 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,085 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,087 copying lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:14,089 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,091 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,094 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,096 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,098 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,100 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,103 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,105 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,107 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,109 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,112 copying lm_eval/tasks/mmlu_prox/it/_mmlu_prox_it.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,114 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,116 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,118 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,121 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,123 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,125 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,127 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,129 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,131 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,133 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,135 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,138 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,140 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,142 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,144 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,146 copying lm_eval/tasks/mmlu_prox/it/_mmlu_prox_lite_it.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,148 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,151 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,153 copying lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:14,155 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,157 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,159 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,161 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,164 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,166 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,168 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,170 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,173 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,175 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,177 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,179 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,181 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,183 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,186 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,188 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,190 copying lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_sr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,192 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,194 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,196 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,199 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,201 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,203 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,205 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,207 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,209 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,212 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,214 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,216 copying lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,218 copying lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_lite_sr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:14,221 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,223 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,225 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,227 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,229 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,232 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,234 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,236 copying lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_wo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,238 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,241 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,243 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,245 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,247 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,250 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,252 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,254 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,256 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,259 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,261 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,263 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,265 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,267 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,269 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,272 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,274 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,276 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,278 copying lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_lite_wo.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,281 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,283 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,285 copying lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:14,287 copying lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_lite_fr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,289 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,291 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,294 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,296 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,298 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,300 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,302 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,305 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,307 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,309 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,311 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,314 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,316 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,318 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,320 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,323 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,325 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,327 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,329 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,331 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,334 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,336 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,338 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,340 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,342 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,344 copying lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_fr.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,346 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,349 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,351 copying lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:14,353 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,355 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,357 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,359 copying lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_lite_cs.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,362 copying lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_cs.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,364 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,366 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,368 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,371 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,373 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,375 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,378 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,380 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,382 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,384 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,387 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,389 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,392 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,394 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,396 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,398 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,400 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,402 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,404 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,407 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,409 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,411 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,413 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,415 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,417 copying lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:14,419 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,422 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,424 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,426 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,429 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,431 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,433 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,436 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,439 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,442 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,445 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,448 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,451 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,453 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,456 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,459 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,462 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,465 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,468 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,471 copying lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_pt.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,474 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,478 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,481 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,484 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,487 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,490 copying lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_lite_pt.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,492 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,495 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,498 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,501 copying lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:14,504 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,506 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,509 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,512 copying lm_eval/tasks/mmlu_prox/id/_mmlu_prox_id.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,515 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,518 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,521 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,523 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,526 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,529 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,532 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,535 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,537 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,539 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,541 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,543 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,546 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,548 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,550 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,552 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,555 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,557 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,559 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,562 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,564 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,566 copying lm_eval/tasks/mmlu_prox/id/_mmlu_prox_lite_id.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,568 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,570 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,572 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,575 copying lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:14,577 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,579 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,581 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,584 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,586 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,588 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,590 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,592 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,595 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,597 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,599 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,601 copying lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_ar.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,603 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,606 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,608 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,611 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,613 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,615 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,617 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,620 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,622 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,624 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,627 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,629 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,631 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,634 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,636 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,638 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,640 copying lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_lite_ar.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,642 copying lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:14,644 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,647 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,649 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,651 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,653 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,655 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,657 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,660 copying lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_lite_ur.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,662 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,664 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,666 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,668 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,671 copying lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_ur.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,673 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,675 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,677 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,680 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,682 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,684 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,687 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,689 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,691 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,694 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,696 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,699 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,701 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,703 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,706 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,708 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,710 copying lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:14,712 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,714 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,716 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,718 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_history.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,721 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,723 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,725 copying lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_lite_hu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,727 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_health.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,730 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,732 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,734 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,736 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,739 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,741 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,744 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,746 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,748 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,750 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_law.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,752 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_biology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,755 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_physics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,757 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_business.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,759 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,762 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,764 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_math.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,766 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,768 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,771 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_economics.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,773 copying lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_hu.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,776 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_other.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,778 copying lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:14,780 creating build/lib/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T15:52:14,782 copying lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml -> build/lib/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T15:52:14,784 creating build/lib/lm_eval/tasks/alghafa/copa_ar 2026-05-11T15:52:14,785 copying lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml -> build/lib/lm_eval/tasks/alghafa/copa_ar 2026-05-11T15:52:14,787 copying lm_eval/tasks/graphwalks/graphwalks_128k.yaml -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T15:52:14,790 copying lm_eval/tasks/graphwalks/graphwalks_1M.yaml -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T15:52:14,792 copying lm_eval/tasks/graphwalks/graphwalks.yaml -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T15:52:14,794 copying lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias_gen.yaml -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:14,796 copying lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias.yaml -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:14,799 copying lm_eval/tasks/catalan_bench/xnli_va.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,801 copying lm_eval/tasks/catalan_bench/openbookqa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,804 copying lm_eval/tasks/catalan_bench/teca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,806 copying lm_eval/tasks/catalan_bench/catcola.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,809 copying lm_eval/tasks/catalan_bench/piqa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,811 copying lm_eval/tasks/catalan_bench/parafraseja.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,813 copying lm_eval/tasks/catalan_bench/mgsm_direct_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,816 copying lm_eval/tasks/catalan_bench/paws_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,818 copying lm_eval/tasks/catalan_bench/xquad_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,820 copying lm_eval/tasks/catalan_bench/arc_ca_easy.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,823 copying lm_eval/tasks/catalan_bench/copa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,825 copying lm_eval/tasks/catalan_bench/coqcat.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,828 copying lm_eval/tasks/catalan_bench/xnli_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,830 copying lm_eval/tasks/catalan_bench/siqa_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,832 copying lm_eval/tasks/catalan_bench/cabreu_abstractive.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,835 copying lm_eval/tasks/catalan_bench/catalan_bench.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,837 copying lm_eval/tasks/catalan_bench/cabreu_extreme.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,839 copying lm_eval/tasks/catalan_bench/cabreu_extractive.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,842 copying lm_eval/tasks/catalan_bench/catalanqa.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,844 copying lm_eval/tasks/catalan_bench/cocoteros_va.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,846 copying lm_eval/tasks/catalan_bench/arc_ca_challenge.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,849 copying lm_eval/tasks/catalan_bench/xstorycloze_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,851 copying lm_eval/tasks/catalan_bench/wnli_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:14,853 copying lm_eval/tasks/catalan_bench/truthfulqa_va/truthfulqa_va.yaml -> build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T15:52:14,855 creating build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:14,857 copying lm_eval/tasks/catalan_bench/phrases_va/phrases_va-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:14,859 copying lm_eval/tasks/catalan_bench/phrases_va/phrases_ca-va.yaml -> build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:14,861 copying lm_eval/tasks/catalan_bench/flores_ca/flores_pt-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,863 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-eu.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,866 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-it.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,868 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-de.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,870 copying lm_eval/tasks/catalan_bench/flores_ca/flores_eu-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,872 copying lm_eval/tasks/catalan_bench/flores_ca/flores_it-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,874 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-es.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,876 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-pt.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,879 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-fr.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,881 copying lm_eval/tasks/catalan_bench/flores_ca/flores_es-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,884 copying lm_eval/tasks/catalan_bench/flores_ca/flores_fr-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,886 copying lm_eval/tasks/catalan_bench/flores_ca/flores_de-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,888 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-en.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,890 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,893 copying lm_eval/tasks/catalan_bench/flores_ca/flores_en-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,895 copying lm_eval/tasks/catalan_bench/flores_ca/flores_gl-ca.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,897 copying lm_eval/tasks/catalan_bench/flores_ca/flores_ca-gl.yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:14,900 creating build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:14,901 copying lm_eval/tasks/bangla/bangla_commonsenseqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:14,903 copying lm_eval/tasks/bangla/bangla_mmlu.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:14,905 copying lm_eval/tasks/bangla/bangla_openbookqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:14,908 copying lm_eval/tasks/bangla/bangla_piqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:14,910 copying lm_eval/tasks/bangla/bangla_boolqa.yaml -> build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:14,912 copying lm_eval/tasks/winogender/winogender_gotcha.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,914 copying lm_eval/tasks/winogender/winogender_gotcha_female.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,917 copying lm_eval/tasks/winogender/winogender_male.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,919 copying lm_eval/tasks/winogender/winogender_neutral.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,922 copying lm_eval/tasks/winogender/winogender.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,924 copying lm_eval/tasks/winogender/winogender_female.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,926 copying lm_eval/tasks/winogender/winogender_gotcha_male.yaml -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:14,928 copying lm_eval/tasks/olaph/olaph_perplexity.yaml -> build/lib/lm_eval/tasks/olaph 2026-05-11T15:52:14,930 copying lm_eval/tasks/olaph/olaph.yaml -> build/lib/lm_eval/tasks/olaph 2026-05-11T15:52:14,932 copying lm_eval/tasks/mediqa_qa2019/mediqa_qa2019_perplexity.yaml -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:14,935 copying lm_eval/tasks/mediqa_qa2019/mediqa_qa2019.yaml -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:14,937 creating build/lib/lm_eval/tasks/pile_10k 2026-05-11T15:52:14,938 copying lm_eval/tasks/pile_10k/pile_10k.yaml -> build/lib/lm_eval/tasks/pile_10k 2026-05-11T15:52:14,940 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,942 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,944 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,946 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,949 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,951 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,953 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,955 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,958 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,960 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,962 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,965 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,967 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,969 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,971 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,974 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,976 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,978 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,980 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,983 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,985 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,987 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,990 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,992 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,994 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,997 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:14,999 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:15,001 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:15,004 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:15,006 copying lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:15,008 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,010 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,013 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,015 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,017 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,020 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,022 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,024 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,026 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,029 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,031 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,033 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,035 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,037 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,040 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,042 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,045 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,047 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,050 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,052 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,054 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,056 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,059 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,062 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,064 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,066 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,068 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,070 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,073 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,075 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,077 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,079 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,082 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,084 copying lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:15,086 copying lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,088 copying lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,091 copying lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,093 copying lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,095 copying lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,097 copying lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,100 copying lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,102 copying lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,104 copying lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,106 copying lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,109 copying lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,111 copying lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,113 copying lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,115 copying lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,118 copying lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,120 copying lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,122 copying lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,124 copying lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,126 copying lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,129 copying lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,131 copying lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,133 copying lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,135 copying lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,137 copying lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,139 copying lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,142 copying lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,144 copying lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,146 copying lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,148 copying lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,150 copying lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,152 copying lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:15,155 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,157 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,159 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,162 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,164 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,166 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,169 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,171 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,173 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,175 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,178 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,180 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,182 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,184 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,187 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,189 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,191 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,194 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,196 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,198 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,200 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,203 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,205 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,207 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,209 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,211 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,214 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,216 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,218 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,221 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,223 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,225 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,227 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,230 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,232 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,234 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,237 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,239 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,241 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,244 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,246 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,248 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,250 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,253 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,255 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,257 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,259 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,262 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,264 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,266 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,268 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,271 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,273 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,275 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,278 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,280 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,283 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,285 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,288 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,294 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,297 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,299 copying lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:15,301 copying lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,305 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,307 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,310 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,312 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,314 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,316 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,318 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,320 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,323 copying lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,325 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,327 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,329 copying lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,331 copying lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,334 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,336 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,338 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,340 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,343 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,345 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,347 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,350 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,352 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,354 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,356 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,359 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,361 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,363 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,365 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,368 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,370 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,372 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,374 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,377 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,379 copying lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,381 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,383 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,386 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,388 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,390 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,393 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,395 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,397 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,399 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,401 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,404 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,406 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,408 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,410 copying lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,413 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,415 copying lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,417 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,419 copying lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,421 copying lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,424 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,426 copying lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,428 copying lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,430 copying lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,432 copying lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,435 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,437 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,439 copying lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:15,442 copying lm_eval/tasks/qasper/freeform.yaml -> build/lib/lm_eval/tasks/qasper 2026-05-11T15:52:15,444 copying lm_eval/tasks/qasper/bool.yaml -> build/lib/lm_eval/tasks/qasper 2026-05-11T15:52:15,446 creating build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,447 copying lm_eval/tasks/aexams/_aexams.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,450 copying lm_eval/tasks/aexams/aexams_Biology.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,452 copying lm_eval/tasks/aexams/aexams_Science.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,454 copying lm_eval/tasks/aexams/aexams_IslamicStudies.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,457 copying lm_eval/tasks/aexams/aexams_Social.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,459 copying lm_eval/tasks/aexams/aexams_Physics.yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:15,462 copying lm_eval/tasks/translation/wmt16_de-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,464 copying lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,466 copying lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,469 copying lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,471 copying lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,473 copying lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,476 copying lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,478 copying lm_eval/tasks/translation/wmt16_en-de.yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:15,480 copying lm_eval/tasks/mutual/mutual.yaml -> build/lib/lm_eval/tasks/mutual 2026-05-11T15:52:15,483 copying lm_eval/tasks/mutual/multual_plus.yaml -> build/lib/lm_eval/tasks/mutual 2026-05-11T15:52:15,485 creating build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,486 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,489 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_other.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,491 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,494 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,496 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,498 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,500 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,503 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,505 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,507 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,510 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,512 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,514 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,517 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,519 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,521 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,524 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,526 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,528 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,531 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,533 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,535 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,537 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,540 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,542 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,544 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,547 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,549 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,551 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,553 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,556 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,558 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,560 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,562 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,565 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,567 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,569 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,572 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,574 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,576 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,579 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,581 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,583 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,586 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,588 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,591 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,593 copying lm_eval/tasks/kmmlu/hard/_kmmlu_hard_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,595 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,598 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,600 copying lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:15,602 creating build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,603 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,606 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,609 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,612 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,614 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,617 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,619 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,623 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,625 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,629 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,631 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,634 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,637 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,640 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,643 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,645 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,648 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,650 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,653 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,656 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,659 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,662 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,665 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,668 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,671 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,674 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,676 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,679 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,682 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,684 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,687 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,690 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,693 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,696 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,699 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,701 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,704 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,707 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,709 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_other.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,711 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,714 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,717 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,720 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,723 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,725 copying lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,728 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,731 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,733 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,736 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,739 copying lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:15,742 creating build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,743 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,746 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,748 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,751 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,753 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,755 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,757 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_other.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,760 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,762 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,764 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,766 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,768 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,770 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,772 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,775 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,777 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,779 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,781 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,783 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,785 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,788 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,790 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,792 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,794 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,797 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,799 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,801 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,803 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,806 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,808 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,810 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,812 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,814 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,816 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,819 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,821 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,823 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,826 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,828 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,830 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,832 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,834 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,837 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,839 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,841 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,844 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,846 copying lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,848 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,850 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,853 copying lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:15,855 creating build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,856 copying lm_eval/tasks/kmmlu/default/kmmlu_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,858 copying lm_eval/tasks/kmmlu/default/kmmlu_law.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,860 copying lm_eval/tasks/kmmlu/default/kmmlu_management.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,862 copying lm_eval/tasks/kmmlu/default/kmmlu_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,864 copying lm_eval/tasks/kmmlu/default/kmmlu_math.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,867 copying lm_eval/tasks/kmmlu/default/kmmlu_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,869 copying lm_eval/tasks/kmmlu/default/kmmlu_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,871 copying lm_eval/tasks/kmmlu/default/kmmlu_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,873 copying lm_eval/tasks/kmmlu/default/_kmmlu_default.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,876 copying lm_eval/tasks/kmmlu/default/_kmmlu_other.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,878 copying lm_eval/tasks/kmmlu/default/kmmlu_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,880 copying lm_eval/tasks/kmmlu/default/kmmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,882 copying lm_eval/tasks/kmmlu/default/kmmlu_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,884 copying lm_eval/tasks/kmmlu/default/kmmlu_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,887 copying lm_eval/tasks/kmmlu/default/kmmlu_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,889 copying lm_eval/tasks/kmmlu/default/kmmlu_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,891 copying lm_eval/tasks/kmmlu/default/kmmlu_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,893 copying lm_eval/tasks/kmmlu/default/kmmlu_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,896 copying lm_eval/tasks/kmmlu/default/kmmlu_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,898 copying lm_eval/tasks/kmmlu/default/kmmlu_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,900 copying lm_eval/tasks/kmmlu/default/kmmlu_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,902 copying lm_eval/tasks/kmmlu/default/kmmlu_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,904 copying lm_eval/tasks/kmmlu/default/kmmlu_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,906 copying lm_eval/tasks/kmmlu/default/kmmlu_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,909 copying lm_eval/tasks/kmmlu/default/kmmlu_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,911 copying lm_eval/tasks/kmmlu/default/_kmmlu_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,913 copying lm_eval/tasks/kmmlu/default/kmmlu_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,915 copying lm_eval/tasks/kmmlu/default/kmmlu_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,917 copying lm_eval/tasks/kmmlu/default/_kmmlu_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,919 copying lm_eval/tasks/kmmlu/default/kmmlu_education.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,921 copying lm_eval/tasks/kmmlu/default/kmmlu_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,924 copying lm_eval/tasks/kmmlu/default/kmmlu_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,926 copying lm_eval/tasks/kmmlu/default/kmmlu_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,928 copying lm_eval/tasks/kmmlu/default/kmmlu_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,930 copying lm_eval/tasks/kmmlu/default/kmmlu_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,932 copying lm_eval/tasks/kmmlu/default/kmmlu_health.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,935 copying lm_eval/tasks/kmmlu/default/kmmlu_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,937 copying lm_eval/tasks/kmmlu/default/kmmlu_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,939 copying lm_eval/tasks/kmmlu/default/kmmlu_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,941 copying lm_eval/tasks/kmmlu/default/kmmlu_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,943 copying lm_eval/tasks/kmmlu/default/kmmlu_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,946 copying lm_eval/tasks/kmmlu/default/kmmlu_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,949 copying lm_eval/tasks/kmmlu/default/kmmlu_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,952 copying lm_eval/tasks/kmmlu/default/kmmlu_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,954 copying lm_eval/tasks/kmmlu/default/_kmmlu_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,957 copying lm_eval/tasks/kmmlu/default/kmmlu_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,960 copying lm_eval/tasks/kmmlu/default/kmmlu_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,962 copying lm_eval/tasks/kmmlu/default/kmmlu_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,965 copying lm_eval/tasks/kmmlu/default/kmmlu_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,968 copying lm_eval/tasks/kmmlu/default/kmmlu_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:15,970 creating build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,971 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,974 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,976 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,979 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,982 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,984 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,987 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,990 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,992 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,995 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:15,997 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,000 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,003 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,006 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,008 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_other.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,011 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,013 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,016 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,019 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,021 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,024 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,027 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,029 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,032 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,035 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,038 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,040 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,043 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_humss.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,045 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,048 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,050 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,052 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,054 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,056 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,058 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,060 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,062 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,064 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,067 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,069 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,071 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_stem.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,073 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,075 copying lm_eval/tasks/kmmlu/direct/_kmmlu_direct_applied_science.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,078 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,080 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,082 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,084 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,086 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,088 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,091 copying lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:16,093 copying lm_eval/tasks/swde/swde.yaml -> build/lib/lm_eval/tasks/swde 2026-05-11T15:52:16,095 creating build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:16,096 copying lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:16,098 copying lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:16,101 copying lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:16,103 copying lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:16,105 copying lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:16,107 copying lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,110 copying lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,112 copying lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,114 copying lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,116 copying lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,118 copying lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,120 copying lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,122 copying lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,124 copying lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,126 copying lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,128 copying lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,130 copying lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,132 copying lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,135 copying lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,137 copying lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,139 copying lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,141 copying lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,144 copying lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,146 copying lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,148 copying lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,150 copying lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,152 copying lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,154 copying lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,156 copying lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,158 copying lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,161 copying lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,163 copying lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,165 copying lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,167 copying lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,169 copying lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,171 copying lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,174 copying lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,176 copying lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,178 copying lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,180 copying lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,182 copying lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,184 copying lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,186 copying lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,189 copying lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,191 copying lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,193 copying lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,195 copying lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,197 copying lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,199 copying lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,201 copying lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,204 copying lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,206 copying lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,208 copying lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,210 copying lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,212 copying lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,214 copying lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,216 copying lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,219 copying lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,221 copying lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,223 copying lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,225 copying lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,227 copying lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,230 copying lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,232 copying lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,234 copying lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,236 copying lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,238 copying lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,240 copying lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,243 copying lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,245 copying lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,247 copying lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,249 copying lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,252 copying lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,254 copying lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,256 copying lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,259 copying lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,261 copying lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,263 copying lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,265 copying lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,267 copying lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,269 copying lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,271 copying lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,274 copying lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,276 copying lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,278 copying lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,280 copying lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,282 copying lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,284 copying lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,286 copying lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,288 copying lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,291 copying lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,293 copying lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,295 copying lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,298 copying lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,300 copying lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,302 copying lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,304 copying lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,306 copying lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,308 copying lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,311 copying lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,313 copying lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,315 copying lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,317 copying lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,320 copying lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,322 copying lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,324 copying lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,326 copying lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,329 copying lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,331 copying lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,333 copying lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,335 copying lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,337 copying lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,339 copying lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,341 copying lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,344 copying lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,346 copying lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,348 copying lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,350 copying lm_eval/tasks/belebele/_belebele.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,352 copying lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,354 copying lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,357 copying lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,359 copying lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,361 copying lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,363 copying lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,365 copying lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,367 copying lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,370 copying lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,372 copying lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:16,374 copying lm_eval/tasks/winogrande/default.yaml -> build/lib/lm_eval/tasks/winogrande 2026-05-11T15:52:16,376 copying lm_eval/tasks/chartqa/chartqa.yaml -> build/lib/lm_eval/tasks/chartqa 2026-05-11T15:52:16,379 copying lm_eval/tasks/chartqa/chartqa_llama.yaml -> build/lib/lm_eval/tasks/chartqa 2026-05-11T15:52:16,381 copying lm_eval/tasks/chartqa/chartqa_llama_90.yaml -> build/lib/lm_eval/tasks/chartqa 2026-05-11T15:52:16,383 copying lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,385 copying lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,388 copying lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,390 copying lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,392 copying lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,394 copying lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,396 copying lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,399 copying lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,401 copying lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,403 copying lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,405 copying lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,408 copying lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,410 copying lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,412 copying lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,414 copying lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,416 copying lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,419 copying lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,421 copying lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,423 copying lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,425 copying lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,427 copying lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,429 copying lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:16,432 copying lm_eval/tasks/click/click.yaml -> build/lib/lm_eval/tasks/click 2026-05-11T15:52:16,434 copying lm_eval/tasks/click/click_lang/click_lang_function.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:16,436 copying lm_eval/tasks/click/click_lang/click_lang_text.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:16,438 copying lm_eval/tasks/click/click_lang/_click_lang.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:16,441 copying lm_eval/tasks/click/click_lang/click_lang_grammar.yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:16,443 copying lm_eval/tasks/click/click_cul/click_cul_geography.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,445 copying lm_eval/tasks/click/click_cul/click_cul_society.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,447 copying lm_eval/tasks/click/click_cul/click_cul_kpop.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,449 copying lm_eval/tasks/click/click_cul/click_cul_politics.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,452 copying lm_eval/tasks/click/click_cul/click_cul_economy.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,454 copying lm_eval/tasks/click/click_cul/_click_cul.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,456 copying lm_eval/tasks/click/click_cul/click_cul_tradition.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,459 copying lm_eval/tasks/click/click_cul/click_cul_history.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,461 copying lm_eval/tasks/click/click_cul/click_cul_law.yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:16,463 copying lm_eval/tasks/kobest/_kobest.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:16,465 copying lm_eval/tasks/kobest/kobest_wic.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:16,468 copying lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:16,470 copying lm_eval/tasks/kobest/kobest_boolq.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:16,472 copying lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:16,475 copying lm_eval/tasks/kobest/kobest_copa.yaml -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:16,477 copying lm_eval/tasks/squad_completion/squad_completion.yaml -> build/lib/lm_eval/tasks/squad_completion 2026-05-11T15:52:16,479 creating build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,480 copying lm_eval/tasks/zhoblimp/fci_renhe_prepP.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,482 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,484 copying lm_eval/tasks/zhoblimp/question_V_not_VP_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,486 copying lm_eval/tasks/zhoblimp/BA_suo_adverbial_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,489 copying lm_eval/tasks/zhoblimp/left_adverbial_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,491 copying lm_eval/tasks/zhoblimp/zhoblimp_group.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,493 copying lm_eval/tasks/zhoblimp/BA_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,495 copying lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,497 copying lm_eval/tasks/zhoblimp/question_particle_nandao.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,500 copying lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,502 copying lm_eval/tasks/zhoblimp/PN_numP_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,504 copying lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,506 copying lm_eval/tasks/zhoblimp/left_adverbial_e.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,508 copying lm_eval/tasks/zhoblimp/noun_adjective_shi.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,510 copying lm_eval/tasks/zhoblimp/anaphor_number_agreement.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,513 copying lm_eval/tasks/zhoblimp/right_yijing_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,515 copying lm_eval/tasks/zhoblimp/npi_renhe_wh_question_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,517 copying lm_eval/tasks/zhoblimp/fci_renhe_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,519 copying lm_eval/tasks/zhoblimp/fci_renhe_suoyou.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,521 copying lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,524 copying lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,526 copying lm_eval/tasks/zhoblimp/intransitive_double_obj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,528 copying lm_eval/tasks/zhoblimp/you_yige.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,530 copying lm_eval/tasks/zhoblimp/question_A_not_A_daodi_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,532 copying lm_eval/tasks/zhoblimp/question_A_not_A.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,534 copying lm_eval/tasks/zhoblimp/left_adverbial_d.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,537 copying lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_locP.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,539 copying lm_eval/tasks/zhoblimp/classifier_noun_agreement_no_gap.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,541 copying lm_eval/tasks/zhoblimp/question_nandao_raising_1_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,543 copying lm_eval/tasks/zhoblimp/control_modal_vs_raising_modal.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,546 copying lm_eval/tasks/zhoblimp/ellipsis_n_bar_class.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,548 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,550 copying lm_eval/tasks/zhoblimp/PN_numP_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,553 copying lm_eval/tasks/zhoblimp/BEI_construction_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,555 copying lm_eval/tasks/zhoblimp/BA_no_progressive.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,557 copying lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,559 copying lm_eval/tasks/zhoblimp/anaphor_gender_agreement.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,561 copying lm_eval/tasks/zhoblimp/relative_operator_intepretation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,564 copying lm_eval/tasks/zhoblimp/ya_insertion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,566 copying lm_eval/tasks/zhoblimp/ellipsis_adj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,568 copying lm_eval/tasks/zhoblimp/BA_meiba.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,570 copying lm_eval/tasks/zhoblimp/question_particle_daodi_choice_tran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,572 copying lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,575 copying lm_eval/tasks/zhoblimp/classifier_noun_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,577 copying lm_eval/tasks/zhoblimp/topicalization_OSV.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,579 copying lm_eval/tasks/zhoblimp/question_nandao_raising_1_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,582 copying lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,584 copying lm_eval/tasks/zhoblimp/npi_renhe_conditional.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,586 copying lm_eval/tasks/zhoblimp/left_dou.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,588 copying lm_eval/tasks/zhoblimp/BA_no_stative_verb.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,590 copying lm_eval/tasks/zhoblimp/adjective_transitive_dui.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,592 copying lm_eval/tasks/zhoblimp/BEI_preposition.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,595 copying lm_eval/tasks/zhoblimp/modal_raising_topicalization.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,597 copying lm_eval/tasks/zhoblimp/ellipsis_double_object.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,599 copying lm_eval/tasks/zhoblimp/BA_inversion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,601 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_tran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,604 copying lm_eval/tasks/zhoblimp/preposition_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,606 copying lm_eval/tasks/zhoblimp/BA_verb_le_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,608 copying lm_eval/tasks/zhoblimp/question_A_not_A_indirect.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,610 copying lm_eval/tasks/zhoblimp/npi_renhe_A_not_A_question.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,612 copying lm_eval/tasks/zhoblimp/principle_A_domain_number.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,615 copying lm_eval/tasks/zhoblimp/causative_shi_ba.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,617 copying lm_eval/tasks/zhoblimp/verb_negation_particle.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,619 copying lm_eval/tasks/zhoblimp/BA_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,622 copying lm_eval/tasks/zhoblimp/passive_no_adj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,624 copying lm_eval/tasks/zhoblimp/question_particle_daodi_choice_intran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,626 copying lm_eval/tasks/zhoblimp/BA_verb_le_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,628 copying lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,630 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,633 copying lm_eval/tasks/zhoblimp/question_nandao_scope_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,635 copying lm_eval/tasks/zhoblimp/plural_cardinal_men_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,637 copying lm_eval/tasks/zhoblimp/agent_animacy_subj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,639 copying lm_eval/tasks/zhoblimp/passive_body_part.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,642 copying lm_eval/tasks/zhoblimp/modal_raising_hui.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,644 copying lm_eval/tasks/zhoblimp/question_daodi_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,646 copying lm_eval/tasks/zhoblimp/npi_renhe_wh_question_obj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,648 copying lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,650 copying lm_eval/tasks/zhoblimp/agent_animacy_passive.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,652 copying lm_eval/tasks/zhoblimp/question_A_not_A_daodi_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,654 copying lm_eval/tasks/zhoblimp/plural_cardinal_men_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,656 copying lm_eval/tasks/zhoblimp/question_V_not_VP_1.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,659 copying lm_eval/tasks/zhoblimp/intransitive_no_obj.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,661 copying lm_eval/tasks/zhoblimp/agent_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,663 copying lm_eval/tasks/zhoblimp/nominal_definite_men.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,665 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_intran.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,667 copying lm_eval/tasks/zhoblimp/BA_BEI_subj_drop.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,670 copying lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,672 copying lm_eval/tasks/zhoblimp/nominal_modal_insertion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,674 copying lm_eval/tasks/zhoblimp/preposition_insertion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,676 copying lm_eval/tasks/zhoblimp/BA_suo_adverbial_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,678 copying lm_eval/tasks/zhoblimp/fci_renhe_ruguo.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,681 copying lm_eval/tasks/zhoblimp/left_adverbial_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,683 copying lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,685 copying lm_eval/tasks/zhoblimp/relative_operator_who.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,687 copying lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,690 copying lm_eval/tasks/zhoblimp/passive_suo.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,692 copying lm_eval/tasks/zhoblimp/question_nandao_raising_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,694 copying lm_eval/tasks/zhoblimp/right_yijing_b.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,696 copying lm_eval/tasks/zhoblimp/BEI_construction_a.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,699 copying lm_eval/tasks/zhoblimp/question_daodi_nandao_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,701 copying lm_eval/tasks/zhoblimp/agent_causative.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,703 copying lm_eval/tasks/zhoblimp/topicalization_SOV.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,705 copying lm_eval/tasks/zhoblimp/classifier_noun_agreement.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,708 copying lm_eval/tasks/zhoblimp/BEI_deletion.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,710 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_long_left.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,712 copying lm_eval/tasks/zhoblimp/noun_phrase_conjunction_jian.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,714 copying lm_eval/tasks/zhoblimp/fci_renhe_dou.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,716 copying lm_eval/tasks/zhoblimp/principle_A_c_command_number.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,718 copying lm_eval/tasks/zhoblimp/principle_A_domain.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,721 copying lm_eval/tasks/zhoblimp/agent_animacy_adv.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,723 copying lm_eval/tasks/zhoblimp/BA_duplicate_argument.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,725 copying lm_eval/tasks/zhoblimp/existential_there_subject_raising.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,727 copying lm_eval/tasks/zhoblimp/question_nandao_raising_3.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,729 copying lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,731 copying lm_eval/tasks/zhoblimp/question_nandao_scope_2.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,734 copying lm_eval/tasks/zhoblimp/passive_intransitive.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,736 copying lm_eval/tasks/zhoblimp/passive_agent_deletion_short.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,738 copying lm_eval/tasks/zhoblimp/question_nandao_negation.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,741 copying lm_eval/tasks/zhoblimp/principle_A_c_command.yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:16,743 copying lm_eval/tasks/wsc273/default.yaml -> build/lib/lm_eval/tasks/wsc273 2026-05-11T15:52:16,745 copying lm_eval/tasks/unitxt/law_stack_exchange.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,747 copying lm_eval/tasks/unitxt/medical_abstracts.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,749 copying lm_eval/tasks/unitxt/claim_stance_topic.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,752 copying lm_eval/tasks/unitxt/yahoo_answers_topics.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,754 copying lm_eval/tasks/unitxt/ag_news.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,756 copying lm_eval/tasks/unitxt/ledgar.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,758 copying lm_eval/tasks/unitxt/banking77.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,761 copying lm_eval/tasks/unitxt/financial_tweets.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,764 copying lm_eval/tasks/unitxt/20_newsgroups.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,766 copying lm_eval/tasks/unitxt/cnn_dailymail.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,768 copying lm_eval/tasks/unitxt/stsb.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,770 copying lm_eval/tasks/unitxt/xsum.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,772 copying lm_eval/tasks/unitxt/unfair_tos.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,775 copying lm_eval/tasks/unitxt/dbpedia_14.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,777 copying lm_eval/tasks/unitxt/atis.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,779 copying lm_eval/tasks/unitxt/argument_topic.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,782 copying lm_eval/tasks/unitxt/doc_vqa.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,784 copying lm_eval/tasks/unitxt/coedit_gec.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,786 copying lm_eval/tasks/unitxt/ethos_binary.yaml -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:16,789 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light 2026-05-11T15:52:16,791 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_mt_hellaswag_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:16,793 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_leaderboard_arabic_mt_hellaswag_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:16,795 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_world_religions_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,798 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_sexuality_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,800 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_jurisprudence_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,802 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_business_ethics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,804 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_security_studies_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,807 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_scenarios_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,809 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_aging_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,811 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_marketing_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,813 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_global_facts_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,816 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_biology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,818 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_public_relations_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,820 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_elementary_mathematics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,822 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_econometrics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,825 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_computer_science_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,827 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_computer_science_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,829 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_international_law_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,831 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_accounting_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,834 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_us_foreign_policy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,836 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_psychology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,838 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,840 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_geography_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,843 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_virology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,845 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_computer_security_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,847 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_us_history_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,849 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_law_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,851 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_logical_fallacies_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,854 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_formal_logic_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,856 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_anatomy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,858 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_disputes_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,860 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_biology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,863 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_mathematics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,865 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_chemistry_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,867 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_conceptual_physics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,870 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_clinical_knowledge_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,872 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_prehistory_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,874 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_physics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,876 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_sociology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,879 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_nutrition_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,881 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,883 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_philosophy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,886 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_mathematics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,888 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_world_history_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,890 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_statistics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,893 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_abstract_algebra_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,895 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_management_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,897 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_european_history_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,900 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_machine_learning_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,902 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_astronomy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,904 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_psychology_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,907 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,909 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_medicine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,912 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_physics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,914 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_medicine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,916 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_chemistry_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,918 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_medical_genetics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,921 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_miscellaneous_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,923 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_electrical_engineering_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,925 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_microeconomics_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:16,928 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_leaderboard_arabic_exams_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:16,930 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_exams_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:16,932 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_leaderboard_arabic_mt_race_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:16,934 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_mt_race_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:16,937 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Bahrain_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,939 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Geography_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,942 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Syria_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,944 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Libya_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,946 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Qatar_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,948 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_United_Arab_Emirates_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,951 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Funeral_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,953 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Food_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,955 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Saudi_Arabia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,957 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Egypt_modern_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,959 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromIslam_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,962 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromByzantium_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,964 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Yemen_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,966 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islamic_law_system_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,968 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Language_Origin_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,971 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromPersia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,973 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,976 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Music_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,978 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mauritania_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,980 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Palestine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,982 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Literature_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,984 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_Education_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,987 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Tunisia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,989 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Comoros_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,991 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_computer_and_phone_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,993 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_daily_life_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,996 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Ancient_Egypt_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:16,998 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arab_Empire_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,000 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Oman_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,003 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_History_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,005 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Lebanon_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,007 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Medicine_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,009 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Art_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,012 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Morocco_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,014 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Algeria_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,016 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromAncientEgypt_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,019 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Clothing_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,021 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Iraq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,023 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ornament_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,025 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Astronomy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,027 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mesopotamia_civilization_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,029 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromRome_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,031 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Math_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,033 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Wedding_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,036 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_communication_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,038 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_branches_and_schools_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,040 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Jordan_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,042 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Philosophy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,044 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ceremony_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,046 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Somalia_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,049 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromGreece_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,051 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Architecture_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,053 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Calligraphy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,055 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,057 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Kuwait_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,059 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromChina_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,061 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Culture_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,063 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_entertainment_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,065 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Sudan_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:17,067 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_leaderboard_arabic_mt_toxigen_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:17,069 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_mt_toxigen_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:17,071 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arabic_mt_copa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:17,073 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arbic_leaderboard_arabic_mt_copa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:17,075 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_leaderboard_arabic_mt_boolq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:17,077 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_mt_boolq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:17,079 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_leaderboard_arabic_mt_openbook_qa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:17,081 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_mt_openbook_qa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:17,083 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_mt_mmlu_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:17,085 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_leaderboard_arabic_mt_mmlu_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:17,087 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_leaderboard_arabic_mt_arc_easy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:17,088 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_mt_arc_easy_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:17,090 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_leaderboard_arabic_mt_piqa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:17,092 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_mt_piqa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:17,094 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_leaderboard_arabic_mt_sciq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:17,096 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_mt_sciq_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:17,098 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_mt_arc_challenge_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:17,100 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_leaderboard_arabic_mt_arc_challenge_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:17,102 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,104 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,106 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_msa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,109 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,110 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,113 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_dialects_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,115 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,117 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_mcq_exams_test_ar_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,119 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,121 copying lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_sentiment_task_light.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:17,123 copying lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,125 copying lm_eval/tasks/minerva_math/minerva_math500.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,127 copying lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,129 copying lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,130 copying lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,132 copying lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,134 copying lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,136 copying lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:17,139 copying lm_eval/tasks/hellaswag/hellaswag.yaml -> build/lib/lm_eval/tasks/hellaswag 2026-05-11T15:52:17,140 creating build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,141 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,144 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,145 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,147 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,149 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,151 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,153 copying lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:17,156 copying lm_eval/tasks/score/score_robustness.yaml -> build/lib/lm_eval/tasks/score 2026-05-11T15:52:17,158 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_aqua_rat.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,160 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lsat_rc.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,162 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_aqua_rat.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,164 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_lr.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,166 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_ar.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,168 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_lr.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,170 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_logiqa_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,172 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_logiqa_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,174 copying lm_eval/tasks/score/agi_eval/score_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,176 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_ar.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,178 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_rc.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,180 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_aqua_rat.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,182 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_math.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,184 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_lr.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,186 copying lm_eval/tasks/score/agi_eval/score_option_order_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,188 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,190 copying lm_eval/tasks/score/agi_eval/score_non_greedy_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,192 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_logiqa_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,194 copying lm_eval/tasks/score/agi_eval/score_prompt_robustness_agieval.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,196 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_math.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,198 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,199 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_en.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,201 copying lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_math.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,203 copying lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lsat_rc.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,205 copying lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_ar.yaml -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:17,207 copying lm_eval/tasks/score/math/prompt_robustness_math_precalc.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,210 copying lm_eval/tasks/score/math/prompt_robustness_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,212 copying lm_eval/tasks/score/math/non_greedy_robustness_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,214 copying lm_eval/tasks/score/math/prompt_robustness_math_geometry.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,216 copying lm_eval/tasks/score/math/non_greedy_robustness_math_geometry.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,218 copying lm_eval/tasks/score/math/score_non_greedy_robustness_math.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,220 copying lm_eval/tasks/score/math/non_greedy_robustness_math_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,222 copying lm_eval/tasks/score/math/non_greedy_robustness_math_precalc.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,224 copying lm_eval/tasks/score/math/non_greedy_robustness_math_num_theory.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,226 copying lm_eval/tasks/score/math/non_greedy_robustness_math_prealgebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,228 copying lm_eval/tasks/score/math/prompt_robustness_math_prealgebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,230 copying lm_eval/tasks/score/math/prompt_robustness_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,232 copying lm_eval/tasks/score/math/prompt_robustness_math_num_theory.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,234 copying lm_eval/tasks/score/math/non_greedy_robustness_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,236 copying lm_eval/tasks/score/math/score_prompt_robustness_math.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,238 copying lm_eval/tasks/score/math/score_robustness_math.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,239 copying lm_eval/tasks/score/math/prompt_robustness_math_algebra.yaml -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:17,241 copying lm_eval/tasks/score/mmlu_pro/score_non_greedy_robustness_mmlu_pro.yaml -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:52:17,243 copying lm_eval/tasks/score/mmlu_pro/score_option_order_robustness_mmlu_pro.yaml -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:52:17,245 copying lm_eval/tasks/score/mmlu_pro/score_prompt_robustness_mmlu_pro.yaml -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:52:17,247 copying lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/lib/lm_eval/tasks/realtoxicityprompts 2026-05-11T15:52:17,249 copying lm_eval/tasks/meqsum/meqsum.yaml -> build/lib/lm_eval/tasks/meqsum 2026-05-11T15:52:17,251 creating build/lib/lm_eval/tasks/bear 2026-05-11T15:52:17,252 copying lm_eval/tasks/bear/bear.yaml -> build/lib/lm_eval/tasks/bear 2026-05-11T15:52:17,254 copying lm_eval/tasks/bear/bear_big.yaml -> build/lib/lm_eval/tasks/bear 2026-05-11T15:52:17,256 creating build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:17,257 copying lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:17,259 copying lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:17,261 copying lm_eval/tasks/gsm8k/gsm8k.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:17,263 copying lm_eval/tasks/gsm8k/gsm8k-cot-llama.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:17,265 copying lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:17,267 copying lm_eval/tasks/libra/ru_babilong_qa3.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,269 copying lm_eval/tasks/libra/librusec_history.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,271 copying lm_eval/tasks/libra/_question_answering_and_multiple_choice.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,273 copying lm_eval/tasks/libra/passkey_with_librusec.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,275 copying lm_eval/tasks/libra/ru_sci_passage_count.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,277 copying lm_eval/tasks/libra/ru_2wikimultihopqa.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,279 copying lm_eval/tasks/libra/ru_sci_abstract_retrieval.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,281 copying lm_eval/tasks/libra/matreshka_yes_no.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,283 copying lm_eval/tasks/libra/matreshka_names.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,285 copying lm_eval/tasks/libra/_complex_reasoning_and_mathematical_problems.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,287 copying lm_eval/tasks/libra/ru_babilong_qa4.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,289 copying lm_eval/tasks/libra/long_context_multiq.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,291 copying lm_eval/tasks/libra/ru_gsm100.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,293 copying lm_eval/tasks/libra/ru_babilong_qa5.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,295 copying lm_eval/tasks/libra/ru_babilong_qa2.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,297 copying lm_eval/tasks/libra/ru_babilong_qa1.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,299 copying lm_eval/tasks/libra/librusec_mhqa.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,301 copying lm_eval/tasks/libra/ru_qasper.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,303 copying lm_eval/tasks/libra/passkey.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,305 copying lm_eval/tasks/libra/_simple_information_retrieval.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,307 copying lm_eval/tasks/libra/_multi_hop_question_answering.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,309 copying lm_eval/tasks/libra/ru_quality.yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:17,311 copying lm_eval/tasks/common_voice/common_voice_en.yaml -> build/lib/lm_eval/tasks/common_voice 2026-05-11T15:52:17,313 copying lm_eval/tasks/glianorex/glianorex_en.yaml -> build/lib/lm_eval/tasks/glianorex 2026-05-11T15:52:17,315 copying lm_eval/tasks/glianorex/glianorex_fr.yaml -> build/lib/lm_eval/tasks/glianorex 2026-05-11T15:52:17,317 copying lm_eval/tasks/glianorex/glianorex.yaml -> build/lib/lm_eval/tasks/glianorex 2026-05-11T15:52:17,319 copying lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,320 copying lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,322 copying lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,324 copying lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,326 copying lm_eval/tasks/hendrycks_math/hendrycks_math500.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,328 copying lm_eval/tasks/hendrycks_math/hendrycks_math.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,330 copying lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,332 copying lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,334 copying lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:17,336 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_biology.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,339 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_other.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,341 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,343 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_physics.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,344 copying lm_eval/tasks/mmlu-pro-plus/_mmlu_pro_plus.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,346 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_history.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,348 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,350 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_business.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,352 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_engineering.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,354 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,356 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_economics.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,359 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,361 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_law.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,362 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_health.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,364 copying lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_math.yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:17,366 creating build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,367 copying lm_eval/tasks/acpbench/boolq_cot_2shot/val.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,369 copying lm_eval/tasks/acpbench/boolq_cot_2shot/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,371 copying lm_eval/tasks/acpbench/boolq_cot_2shot/land.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,374 copying lm_eval/tasks/acpbench/boolq_cot_2shot/reach.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,376 copying lm_eval/tasks/acpbench/boolq_cot_2shot/prog.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,378 copying lm_eval/tasks/acpbench/boolq_cot_2shot/just.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,380 copying lm_eval/tasks/acpbench/boolq_cot_2shot/app.yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:17,382 copying lm_eval/tasks/acpbench/gen_2shot/next_act.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,384 copying lm_eval/tasks/acpbench/gen_2shot/val.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,387 copying lm_eval/tasks/acpbench/gen_2shot/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,389 copying lm_eval/tasks/acpbench/gen_2shot/land.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,392 copying lm_eval/tasks/acpbench/gen_2shot/reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,394 copying lm_eval/tasks/acpbench/gen_2shot/prog.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,396 copying lm_eval/tasks/acpbench/gen_2shot/just.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,399 copying lm_eval/tasks/acpbench/gen_2shot/app.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:17,401 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/next_act.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,404 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/val.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,407 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,410 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/land.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,413 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/reach.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,416 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/prog.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,418 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/just.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,421 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/app.yaml -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:17,424 creating build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,425 copying lm_eval/tasks/acpbench/mcq_cot_2shot/val.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,427 copying lm_eval/tasks/acpbench/mcq_cot_2shot/act_reach.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,430 copying lm_eval/tasks/acpbench/mcq_cot_2shot/land.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,432 copying lm_eval/tasks/acpbench/mcq_cot_2shot/reach.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,434 copying lm_eval/tasks/acpbench/mcq_cot_2shot/prog.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,436 copying lm_eval/tasks/acpbench/mcq_cot_2shot/just.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,439 copying lm_eval/tasks/acpbench/mcq_cot_2shot/app.yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:17,441 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_morocco.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,444 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_nile_valley.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,446 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,448 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_egypt.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,450 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_gulf.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,452 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_palestine.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,454 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_libya.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,456 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_levant.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,458 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_uae.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,461 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_jordan.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,463 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_sudan.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,465 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_algeria.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,467 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_lebanon.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,470 copying lm_eval/tasks/arab_culture_completion/_arab_culture_completion_north_africa.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,472 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_tunisia.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,474 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_yemen.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,477 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_syria.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,479 copying lm_eval/tasks/arab_culture_completion/arab_culture_completion_ksa.yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:17,481 creating build/lib/lm_eval/tasks/storycloze 2026-05-11T15:52:17,482 copying lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/lib/lm_eval/tasks/storycloze 2026-05-11T15:52:17,485 copying lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/lib/lm_eval/tasks/storycloze 2026-05-11T15:52:17,487 copying lm_eval/tasks/basque_bench/arc_eu_easy.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,489 copying lm_eval/tasks/basque_bench/arc_eu_challenge.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,492 copying lm_eval/tasks/basque_bench/basque_bench.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,494 copying lm_eval/tasks/basque_bench/wnli_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,496 copying lm_eval/tasks/basque_bench/mgsm_native_cot_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,498 copying lm_eval/tasks/basque_bench/mgsm_direct_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,500 copying lm_eval/tasks/basque_bench/xcopa_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,503 copying lm_eval/tasks/basque_bench/paws_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,505 copying lm_eval/tasks/basque_bench/piqa_eu.yaml -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:17,507 copying lm_eval/tasks/basque_bench/flores_eu/flores_fr-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,509 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-it.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,511 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-fr.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,513 copying lm_eval/tasks/basque_bench/flores_eu/flores_gl-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,515 copying lm_eval/tasks/basque_bench/flores_eu/flores_es-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,517 copying lm_eval/tasks/basque_bench/flores_eu/flores_ca-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,520 copying lm_eval/tasks/basque_bench/flores_eu/flores_en-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,522 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-gl.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,524 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-es.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,526 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-de.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,528 copying lm_eval/tasks/basque_bench/flores_eu/flores_pt-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,531 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-ca.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,533 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-pt.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,535 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu-en.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,537 copying lm_eval/tasks/basque_bench/flores_eu/flores_de-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,539 copying lm_eval/tasks/basque_bench/flores_eu/flores_it-eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,541 copying lm_eval/tasks/basque_bench/flores_eu/flores_eu.yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:17,544 copying lm_eval/tasks/darijammlu/darijammlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,546 copying lm_eval/tasks/darijammlu/darijammlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,548 copying lm_eval/tasks/darijammlu/darijammlu_management_ar.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,550 copying lm_eval/tasks/darijammlu/darijammlu_geography.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,552 copying lm_eval/tasks/darijammlu/darijammlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,554 copying lm_eval/tasks/darijammlu/darijammlu_international_law.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,557 copying lm_eval/tasks/darijammlu/darijammlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,559 copying lm_eval/tasks/darijammlu/darijammlu_public_relations.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,561 copying lm_eval/tasks/darijammlu/darijammlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,563 copying lm_eval/tasks/darijammlu/darijammlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,565 copying lm_eval/tasks/darijammlu/darijammlu_nutrition.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,567 copying lm_eval/tasks/darijammlu/darijammlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,570 copying lm_eval/tasks/darijammlu/darijammlu_economics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,572 copying lm_eval/tasks/darijammlu/darijammlu_biology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,574 copying lm_eval/tasks/darijammlu/darijammlu_civics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,576 copying lm_eval/tasks/darijammlu/darijammlu_physics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,578 copying lm_eval/tasks/darijammlu/darijammlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,580 copying lm_eval/tasks/darijammlu/darijammlu_arabic_language.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,582 copying lm_eval/tasks/darijammlu/darijammlu_sociology.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,584 copying lm_eval/tasks/darijammlu/darijammlu_professional_law.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,586 copying lm_eval/tasks/darijammlu/darijammlu_political_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,589 copying lm_eval/tasks/darijammlu/darijammlu_management.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,591 copying lm_eval/tasks/darijammlu/darijammlu_human_aging.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,593 copying lm_eval/tasks/darijammlu/darijammlu_math.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,596 copying lm_eval/tasks/darijammlu/darijammlu_driving_test.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,598 copying lm_eval/tasks/darijammlu/darijammlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,601 copying lm_eval/tasks/darijammlu/darijammlu_arabic_language_(general).yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,603 copying lm_eval/tasks/darijammlu/darijammlu_law.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,605 copying lm_eval/tasks/darijammlu/_darijammlu_mmlu.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,607 copying lm_eval/tasks/darijammlu/_darijammlu_ar_mmlu.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,610 copying lm_eval/tasks/darijammlu/darijammlu_security_studies.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,612 copying lm_eval/tasks/darijammlu/darijammlu_global_facts.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,614 copying lm_eval/tasks/darijammlu/darijammlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,616 copying lm_eval/tasks/darijammlu/darijammlu_philosophy_ar.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,619 copying lm_eval/tasks/darijammlu/darijammlu_marketing.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,621 copying lm_eval/tasks/darijammlu/darijammlu_arabic_language_(grammar).yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,624 copying lm_eval/tasks/darijammlu/darijammlu_history.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,626 copying lm_eval/tasks/darijammlu/darijammlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,628 copying lm_eval/tasks/darijammlu/_darijammlu.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,630 copying lm_eval/tasks/darijammlu/darijammlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,633 copying lm_eval/tasks/darijammlu/darijammlu_natural_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,635 copying lm_eval/tasks/darijammlu/darijammlu_social_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,637 copying lm_eval/tasks/darijammlu/darijammlu_computer_science.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,639 copying lm_eval/tasks/darijammlu/darijammlu_philosophy.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,641 copying lm_eval/tasks/darijammlu/darijammlu_accounting.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,644 copying lm_eval/tasks/darijammlu/darijammlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,646 copying lm_eval/tasks/darijammlu/darijammlu_world_religions.yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:17,648 copying lm_eval/tasks/jsonschema_bench/jsonschema_bench_hard.yaml -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T15:52:17,650 copying lm_eval/tasks/jsonschema_bench/jsonschema_bench_easy.yaml -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T15:52:17,653 copying lm_eval/tasks/jsonschema_bench/jsonschema_bench_medium.yaml -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T15:52:17,655 copying lm_eval/tasks/discrim_eval/discrim_eval_implicit.yaml -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T15:52:17,657 copying lm_eval/tasks/discrim_eval/discrim_eval_explicit.yaml -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T15:52:17,660 copying lm_eval/tasks/super_glue/cb/default.yaml -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T15:52:17,662 copying lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/cb 2026-05-11T15:52:17,664 copying lm_eval/tasks/super_glue/copa/default.yaml -> build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T15:52:17,666 copying lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/copa 2026-05-11T15:52:17,669 creating build/lib/lm_eval/tasks/super_glue/wic 2026-05-11T15:52:17,670 copying lm_eval/tasks/super_glue/wic/default.yaml -> build/lib/lm_eval/tasks/super_glue/wic 2026-05-11T15:52:17,672 copying lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wic 2026-05-11T15:52:17,674 copying lm_eval/tasks/super_glue/record/default.yaml -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T15:52:17,677 copying lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/record 2026-05-11T15:52:17,679 copying lm_eval/tasks/super_glue/multirc/default.yaml -> build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T15:52:17,681 copying lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/multirc 2026-05-11T15:52:17,684 creating build/lib/lm_eval/tasks/super_glue/rte 2026-05-11T15:52:17,685 copying lm_eval/tasks/super_glue/rte/default.yaml -> build/lib/lm_eval/tasks/super_glue/rte 2026-05-11T15:52:17,687 copying lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/rte 2026-05-11T15:52:17,689 copying lm_eval/tasks/super_glue/wsc/default.yaml -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T15:52:17,691 copying lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/wsc 2026-05-11T15:52:17,694 creating build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T15:52:17,695 copying lm_eval/tasks/super_glue/boolq/default.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T15:52:17,697 copying lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T15:52:17,700 copying lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/lib/lm_eval/tasks/super_glue/boolq 2026-05-11T15:52:17,702 copying lm_eval/tasks/hrm8k/en/hrm8k_omni_math_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:17,704 copying lm_eval/tasks/hrm8k/en/hrm8k_mmmlu_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:17,707 copying lm_eval/tasks/hrm8k/en/hrm8k_ksm_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:17,709 copying lm_eval/tasks/hrm8k/en/hrm8k_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:17,711 copying lm_eval/tasks/hrm8k/en/hrm8k_gsm8k_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:17,714 copying lm_eval/tasks/hrm8k/en/hrm8k_math_en.yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:17,716 copying lm_eval/tasks/hrm8k/default/hrm8k_ksm.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:17,718 copying lm_eval/tasks/hrm8k/default/hrm8k_math.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:17,720 copying lm_eval/tasks/hrm8k/default/hrm8k_omni_math.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:17,722 copying lm_eval/tasks/hrm8k/default/hrm8k_gsm8k.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:17,724 copying lm_eval/tasks/hrm8k/default/hrm8k.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:17,727 copying lm_eval/tasks/hrm8k/default/hrm8k_mmmlu.yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:17,729 copying lm_eval/tasks/toxigen/toxigen.yaml -> build/lib/lm_eval/tasks/toxigen 2026-05-11T15:52:17,731 copying lm_eval/tasks/icelandic_winogrande/default.yaml -> build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T15:52:17,733 copying lm_eval/tasks/darijahellaswag/darijahellaswag.yaml -> build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T15:52:17,735 creating build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:17,736 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-llama.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:17,739 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:17,742 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:17,744 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-zeroshot.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:17,746 copying lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-self-consistency.yaml -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:17,748 copying lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T15:52:17,751 copying lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T15:52:17,753 copying lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T15:52:17,755 copying lm_eval/tasks/bbq/bbq_generate_ambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:17,758 copying lm_eval/tasks/bbq/bbq_multiple_choice_ambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:17,760 copying lm_eval/tasks/bbq/bbq_multiple_choice_disambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:17,762 copying lm_eval/tasks/bbq/bbq_generate.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:17,765 copying lm_eval/tasks/bbq/bbq_multiple_choice.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:17,767 copying lm_eval/tasks/bbq/bbq_generate_disambig.yaml -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:17,769 creating build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,770 copying lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,773 copying lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,775 copying lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,778 copying lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,780 copying lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,782 copying lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,784 copying lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,787 copying lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,789 copying lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,791 copying lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,794 copying lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,796 copying lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,798 copying lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,800 copying lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,802 copying lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,805 copying lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,807 copying lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,810 copying lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,812 copying lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,814 copying lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,817 copying lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,819 copying lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,821 copying lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,823 copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,826 copying lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,828 copying lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,830 copying lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,833 copying lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,835 copying lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,837 copying lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,840 copying lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,842 copying lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,844 copying lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,847 copying lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,849 copying lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,851 copying lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,854 copying lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,856 copying lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,858 copying lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,860 copying lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,862 copying lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,865 copying lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,867 copying lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,869 copying lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,871 copying lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,873 copying lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,876 copying lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,878 copying lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,880 copying lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,882 copying lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,885 copying lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,887 copying lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,889 copying lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,891 copying lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,894 copying lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,896 copying lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,898 copying lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,900 copying lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,903 copying lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,905 copying lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,907 copying lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,909 copying lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,911 copying lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,913 copying lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,915 copying lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,918 copying lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,920 copying lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,922 copying lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,924 copying lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,926 copying lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,928 copying lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,930 copying lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,933 copying lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,935 copying lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,937 copying lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,939 copying lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,942 copying lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,944 copying lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,946 copying lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,949 copying lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,951 copying lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,953 copying lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,955 copying lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,958 copying lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,960 copying lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,962 copying lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,964 copying lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,966 copying lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,969 copying lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,971 copying lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,973 copying lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,975 copying lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,977 copying lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,979 copying lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,982 copying lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,984 copying lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,987 copying lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,990 copying lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,992 copying lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,994 copying lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,996 copying lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:17,999 copying lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,001 copying lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,003 copying lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,005 copying lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,008 copying lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,010 copying lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,012 copying lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,014 copying lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,017 copying lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,019 copying lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,021 copying lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,024 copying lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,026 copying lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,030 copying lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,032 copying lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,034 copying lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,037 copying lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,039 copying lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:52:18,040 creating build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,041 copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,044 copying lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,046 copying lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,048 copying lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,050 copying lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,052 copying lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,054 copying lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,056 copying lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,058 copying lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,060 copying lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,063 copying lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,065 copying lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,068 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,071 copying lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,074 copying lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,076 copying lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,079 copying lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,081 copying lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,083 copying lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,086 copying lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,088 copying lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,090 copying lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,093 copying lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,095 copying lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,097 copying lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,099 copying lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,102 copying lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,104 copying lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,106 copying lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,109 copying lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,111 copying lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,114 copying lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,116 copying lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,118 copying lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,120 copying lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,123 copying lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,125 copying lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,127 copying lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,130 copying lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,132 copying lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,134 copying lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,137 copying lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,139 copying lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,141 copying lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,144 copying lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,146 copying lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,148 copying lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,151 copying lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,153 copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,155 copying lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,157 copying lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,160 copying lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,162 copying lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,164 copying lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,167 copying lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,169 copying lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,171 copying lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,173 copying lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,175 copying lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,178 copying lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,180 copying lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,183 copying lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,185 copying lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,187 copying lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,189 copying lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,192 copying lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,194 copying lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,196 copying lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,198 copying lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,201 copying lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,203 copying lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,205 copying lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,208 copying lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,210 copying lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,212 copying lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,214 copying lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,217 copying lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,220 copying lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,222 copying lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,224 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,227 copying lm_eval/tasks/bigbench/generate_until/color.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,229 copying lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,231 copying lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,234 copying lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,236 copying lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,238 copying lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,240 copying lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,242 copying lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,245 copying lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,247 copying lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,249 copying lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,251 copying lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,253 copying lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,256 copying lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,258 copying lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,260 copying lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,262 copying lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,264 copying lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,267 copying lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,269 copying lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,271 copying lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,274 copying lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,276 copying lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,278 copying lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,280 copying lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,283 copying lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,285 copying lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,287 copying lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,290 copying lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,292 copying lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,294 copying lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,296 copying lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,298 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,300 copying lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,303 copying lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,305 copying lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,307 copying lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,309 copying lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,311 copying lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,314 copying lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,316 copying lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,318 copying lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,320 copying lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,322 copying lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,324 copying lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,327 copying lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,329 copying lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,331 copying lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,334 copying lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,336 copying lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,338 copying lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,340 copying lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,343 copying lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,345 copying lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,347 copying lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,349 copying lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,351 copying lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,354 copying lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,356 copying lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,358 copying lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,360 copying lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,362 copying lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,364 copying lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,366 copying lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,368 copying lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,371 copying lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,373 copying lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,375 copying lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,377 copying lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,379 copying lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,382 copying lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,384 copying lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,386 copying lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,389 copying lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,391 copying lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,393 copying lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,396 copying lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,398 copying lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,400 copying lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,403 copying lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,405 copying lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,407 copying lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,410 copying lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,412 copying lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,414 copying lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,417 copying lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,419 copying lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/lib/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:52:18,422 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_complete.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T15:52:18,424 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_leaderboard_arabic_mt_race.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:52:18,426 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_mt_race.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:52:18,428 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_mt_arc_easy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:52:18,430 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_leaderboard_arabic_mt_arc_easy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:52:18,432 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,435 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,437 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,440 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,442 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,444 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,446 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,449 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,451 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,453 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,455 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_marketing.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,457 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,460 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,462 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,464 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,466 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,468 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,470 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,472 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,474 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,476 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,478 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,481 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,483 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_sociology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,485 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,487 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,490 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,492 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,494 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,496 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,499 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,501 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,503 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,506 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,508 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,510 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,512 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,514 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,517 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,519 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,523 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,526 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_international_law.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,528 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,530 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,533 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,535 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,537 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,540 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_management.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,542 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,544 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_virology.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,546 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,549 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,551 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,553 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,555 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,558 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,560 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,562 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:52:18,565 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_mt_mmlu.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:52:18,567 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_leaderboard_arabic_mt_mmlu.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:52:18,569 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_mt_arc_challenge.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:52:18,571 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_leaderboard_arabic_mt_arc_challenge.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:52:18,574 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_leaderboard_arabic_mt_toxigen.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:52:18,576 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_mt_toxigen.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:52:18,578 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_leaderboard_arabic_mt_boolq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:52:18,580 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_mt_boolq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:52:18,582 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_mt_sciq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:52:18,585 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_leaderboard_arabic_mt_sciq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:52:18,587 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ceremony.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,589 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mauritania.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,592 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Jordan.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,594 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_United_Arab_Emirates.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,596 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,599 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Comoros.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,601 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Math.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,603 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromIslam.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,606 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Astronomy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,608 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Lebanon.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,610 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Music.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,612 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Medicine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,615 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Art.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,617 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Palestine.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,619 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Calligraphy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,621 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Tunisia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,623 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Architecture.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,626 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arab_Empire.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,628 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Iraq.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,630 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,633 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Somalia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,635 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Food.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,638 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Libya.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,640 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromAncientEgypt.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,642 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Oman.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,645 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromGreece.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,647 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mesopotamia_civilization.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,649 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_communication.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,651 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Bahrain.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,654 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islamic_law_system.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,656 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Language_Origin.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,658 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromByzantium.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,660 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_Education.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,663 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_branches_and_schools.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,665 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromChina.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,667 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromPersia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,669 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_entertainment.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,672 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_daily_life.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,674 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_History.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,676 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Saudi_Arabia.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,679 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromRome.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,681 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Wedding.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,683 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Philosophy.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,686 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Culture.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,688 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Qatar.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,690 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Sudan.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,693 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Ancient_Egypt.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,696 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Literature.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,698 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Syria.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,700 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Morocco.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,702 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_computer_and_phone.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,705 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Yemen.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,707 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Clothing.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,709 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Geography.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,712 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Egypt_modern.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,714 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Funeral.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,716 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ornament.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,719 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Kuwait.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,721 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Algeria.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:52:18,723 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_mt_hellaswag.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:52:18,726 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_leaderboard_arabic_mt_hellaswag.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:52:18,728 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_mt_piqa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:52:18,730 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_leaderboard_arabic_mt_piqa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:52:18,733 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_leaderboard_arabic_mt_openbook_qa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:52:18,735 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_mt_openbook_qa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:52:18,737 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_exams.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:52:18,739 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_leaderboard_arabic_exams.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:52:18,741 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_mt_copa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:52:18,744 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_leaderboard_arabic_mt_copa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:52:18,746 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,748 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_dialects.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,751 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_sentiment_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,753 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_mcq_exams_test_ar.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,755 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,758 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,760 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,762 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,764 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,767 copying lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_msa.yaml -> build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:52:18,769 copying lm_eval/tasks/medqa/medqa.yaml -> build/lib/lm_eval/tasks/medqa 2026-05-11T15:52:18,771 copying lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/lib/lm_eval/tasks/pubmedqa 2026-05-11T15:52:18,773 copying lm_eval/tasks/scrolls/scrolls_govreport.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,776 copying lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,778 copying lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,780 copying lm_eval/tasks/scrolls/scrolls_qmsum.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,782 copying lm_eval/tasks/scrolls/scrolls_qasper.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,785 copying lm_eval/tasks/scrolls/scrolls_contractnli.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,787 copying lm_eval/tasks/scrolls/scrolls_quality.yaml -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:18,789 copying lm_eval/tasks/mlqa/mlqa_zh_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,791 copying lm_eval/tasks/mlqa/mlqa_zh_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,793 copying lm_eval/tasks/mlqa/mlqa_ar_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,796 copying lm_eval/tasks/mlqa/mlqa_de_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,798 copying lm_eval/tasks/mlqa/mlqa_de_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,800 copying lm_eval/tasks/mlqa/mlqa_de_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,802 copying lm_eval/tasks/mlqa/mlqa_hi_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,804 copying lm_eval/tasks/mlqa/mlqa_es_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,806 copying lm_eval/tasks/mlqa/mlqa_de_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,809 copying lm_eval/tasks/mlqa/mlqa_en_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,811 copying lm_eval/tasks/mlqa/mlqa_es_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,813 copying lm_eval/tasks/mlqa/mlqa_en_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,815 copying lm_eval/tasks/mlqa/mlqa_de_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,817 copying lm_eval/tasks/mlqa/mlqa_zh_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,820 copying lm_eval/tasks/mlqa/mlqa_vi_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,822 copying lm_eval/tasks/mlqa/mlqa_zh_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,824 copying lm_eval/tasks/mlqa/mlqa_en_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,826 copying lm_eval/tasks/mlqa/mlqa_vi_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,829 copying lm_eval/tasks/mlqa/mlqa_es_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,831 copying lm_eval/tasks/mlqa/mlqa_ar_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,833 copying lm_eval/tasks/mlqa/mlqa_hi_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,835 copying lm_eval/tasks/mlqa/mlqa_hi_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,838 copying lm_eval/tasks/mlqa/mlqa_vi_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,840 copying lm_eval/tasks/mlqa/mlqa_es_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,842 copying lm_eval/tasks/mlqa/mlqa_en_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,844 copying lm_eval/tasks/mlqa/mlqa_ar_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,846 copying lm_eval/tasks/mlqa/mlqa_hi_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,849 copying lm_eval/tasks/mlqa/mlqa_zh_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,851 copying lm_eval/tasks/mlqa/mlqa_hi_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,853 copying lm_eval/tasks/mlqa/mlqa_hi_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,855 copying lm_eval/tasks/mlqa/mlqa_vi_es.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,857 copying lm_eval/tasks/mlqa/mlqa_es_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,860 copying lm_eval/tasks/mlqa/mlqa_es_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,862 copying lm_eval/tasks/mlqa/mlqa_de_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,864 copying lm_eval/tasks/mlqa/mlqa_en_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,866 copying lm_eval/tasks/mlqa/mlqa_vi_hi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,868 copying lm_eval/tasks/mlqa/mlqa_zh_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,871 copying lm_eval/tasks/mlqa/mlqa_es_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,873 copying lm_eval/tasks/mlqa/mlqa_ar_vi.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,875 copying lm_eval/tasks/mlqa/mlqa_ar_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,877 copying lm_eval/tasks/mlqa/mlqa_zh_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,879 copying lm_eval/tasks/mlqa/mlqa_en_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,881 copying lm_eval/tasks/mlqa/mlqa_ar_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,883 copying lm_eval/tasks/mlqa/mlqa_en_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,885 copying lm_eval/tasks/mlqa/mlqa_ar_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,887 copying lm_eval/tasks/mlqa/mlqa_hi_de.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,890 copying lm_eval/tasks/mlqa/mlqa_de_zh.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,892 copying lm_eval/tasks/mlqa/mlqa_vi_en.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,894 copying lm_eval/tasks/mlqa/mlqa_vi_ar.yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:18,897 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:52:18,898 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:52:18,901 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:52:18,903 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:52:18,905 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:52:18,907 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:52:18,908 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:52:18,911 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:52:18,913 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:52:18,915 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:52:18,917 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:52:18,918 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:52:18,921 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:52:18,923 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:52:18,925 copying lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:52:18,928 creating build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:52:18,929 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:52:18,931 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:52:18,933 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:52:18,936 copying lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:52:18,938 creating build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:52:18,939 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:52:18,941 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:52:18,943 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:52:18,946 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:52:18,948 copying lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:52:18,950 creating build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:52:18,951 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:52:18,953 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:52:18,955 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:52:18,957 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:52:18,960 copying lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:52:18,962 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:18,964 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:18,966 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:18,969 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:18,971 copying lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:52:18,973 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:18,975 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:18,978 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:18,980 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:18,982 copying lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:52:18,984 creating build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:52:18,985 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p1.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:52:18,988 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p4.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:52:18,990 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p0.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:52:18,992 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p2.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:52:18,994 copying lm_eval/tasks/noreval/norec/norec_document/norec_document_p3.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:52:18,996 creating build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:52:18,997 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p3.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:52:19,000 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p2.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:52:19,002 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p4.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:52:19,004 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p0.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:52:19,006 copying lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p1.yaml -> build/lib/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:52:19,008 creating build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:52:19,009 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:52:19,012 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:52:19,014 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:52:19,016 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:52:19,018 copying lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:52:19,020 creating build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:52:19,021 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:52:19,024 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:52:19,026 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:52:19,028 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:52:19,031 copying lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:52:19,033 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:19,035 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:19,037 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:19,040 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:19,043 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:52:19,047 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:19,051 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:19,054 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:19,057 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:19,060 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:52:19,062 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:52:19,063 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:52:19,067 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:52:19,070 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:52:19,074 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:52:19,077 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:52:19,081 creating build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:52:19,082 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:52:19,086 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:52:19,088 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:52:19,092 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:52:19,096 copying lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:52:19,099 creating build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,100 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,102 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,104 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,106 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p5.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,109 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,111 copying lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:52:19,113 creating build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,114 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,116 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,119 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,121 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p5.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,123 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,125 copying lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:52:19,128 copying lm_eval/tasks/noreval/norquad/norquad_p0.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:19,130 copying lm_eval/tasks/noreval/norquad/norquad_p1.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:19,132 copying lm_eval/tasks/noreval/norquad/norquad_p4.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:19,134 copying lm_eval/tasks/noreval/norquad/norquad_p3.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:19,136 copying lm_eval/tasks/noreval/norquad/norquad_p2.yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:19,138 creating build/lib/lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T15:52:19,139 copying lm_eval/tasks/noreval/norsummarize-instruct/norsummarize_instruct.yaml -> build/lib/lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T15:52:19,142 creating build/lib/lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T15:52:19,143 copying lm_eval/tasks/noreval/norrewrite-instruct/norrewrite_instruct.yaml -> build/lib/lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T15:52:19,145 creating build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:52:19,146 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p0.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:52:19,149 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p4.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:52:19,151 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p2.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:52:19,153 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p1.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:52:19,156 copying lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p3.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:52:19,158 creating build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:52:19,159 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p0.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:52:19,161 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p1.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:52:19,163 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p2.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:52:19,165 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p4.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:52:19,168 copying lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p3.yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:52:19,170 creating build/lib/lm_eval/tasks/noreval/ncb 2026-05-11T15:52:19,171 copying lm_eval/tasks/noreval/ncb/ncb.yaml -> build/lib/lm_eval/tasks/noreval/ncb 2026-05-11T15:52:19,173 creating build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:19,174 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p4.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:19,176 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p3.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:19,178 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p2.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:19,181 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p1.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:19,183 copying lm_eval/tasks/noreval/norbelebele/norbelebele_p0.yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:19,186 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p1.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:19,189 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p4.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:19,191 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p3.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:19,193 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p0.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:19,195 copying lm_eval/tasks/noreval/ask_gec/ask_gec_p2.yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:19,198 copying lm_eval/tasks/eus_trivia/eus_trivia.yaml -> build/lib/lm_eval/tasks/eus_trivia 2026-05-11T15:52:19,200 creating build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,201 copying lm_eval/tasks/ulqa/celep1.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,203 copying lm_eval/tasks/ulqa/celep2.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,205 copying lm_eval/tasks/ulqa/lambada_uyghur.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,207 copying lm_eval/tasks/ulqa/uleval.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,209 copying lm_eval/tasks/ulqa/ulqa_.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,211 copying lm_eval/tasks/ulqa/ulqa.yaml -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:19,214 creating build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,215 copying lm_eval/tasks/ulqa/ulut/wsm.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,217 copying lm_eval/tasks/ulqa/ulut/wag.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,219 copying lm_eval/tasks/ulqa/ulut/wum.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,222 copying lm_eval/tasks/ulqa/ulut/ulut.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,224 copying lm_eval/tasks/ulqa/ulut/nug.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,226 copying lm_eval/tasks/ulqa/ulut/wub.yaml -> build/lib/lm_eval/tasks/ulqa/ulut 2026-05-11T15:52:19,229 copying lm_eval/tasks/fda/fda.yaml -> build/lib/lm_eval/tasks/fda 2026-05-11T15:52:19,232 copying lm_eval/tasks/coqa/default.yaml -> build/lib/lm_eval/tasks/coqa 2026-05-11T15:52:19,234 creating build/lib/lm_eval/tasks/xnli_eu 2026-05-11T15:52:19,235 copying lm_eval/tasks/xnli_eu/xnli_eu.yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T15:52:19,238 copying lm_eval/tasks/xnli_eu/xnli_eu_native.yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T15:52:19,240 copying lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T15:52:19,242 copying lm_eval/tasks/medmcqa/medmcqa.yaml -> build/lib/lm_eval/tasks/medmcqa 2026-05-11T15:52:19,245 copying lm_eval/tasks/ruler/fwe.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,247 copying lm_eval/tasks/ruler/niah_multikey_1.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,250 copying lm_eval/tasks/ruler/qa_squad.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,252 copying lm_eval/tasks/ruler/niah_multivalue.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,254 copying lm_eval/tasks/ruler/vt.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,257 copying lm_eval/tasks/ruler/ruler.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,259 copying lm_eval/tasks/ruler/niah_multikey_3.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,261 copying lm_eval/tasks/ruler/niah_single_3.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,264 copying lm_eval/tasks/ruler/niah_multikey_2.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,267 copying lm_eval/tasks/ruler/niah_multiquery.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,269 copying lm_eval/tasks/ruler/niah_single_2.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,272 copying lm_eval/tasks/ruler/qa_hotpot.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,275 copying lm_eval/tasks/ruler/niah_single_1.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,278 copying lm_eval/tasks/ruler/cwe.yaml -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:19,280 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jnli.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,282 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jaqket_v2.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,285 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jsquad.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,287 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,289 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,291 copying lm_eval/tasks/japanese_leaderboard/_ja_leaderboard.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,294 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,296 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_marc_ja.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,298 copying lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.yaml -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:19,300 creating build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:52:19,301 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:52:19,304 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:52:19,306 copying lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/lib/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:52:19,308 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,311 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,313 copying lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,315 copying lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,317 copying lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,320 copying lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,322 copying lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,324 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,326 copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,329 copying lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,331 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,333 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,335 copying lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,338 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,340 copying lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,342 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,345 copying lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,347 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,349 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,351 copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,354 copying lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,356 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,358 copying lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,361 copying lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,363 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,365 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,367 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,370 copying lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,372 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,374 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,377 copying lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,379 copying lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,381 copying lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,384 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,386 copying lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,388 copying lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,391 copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,393 copying lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,395 copying lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,398 copying lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,400 copying lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,402 copying lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,405 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,407 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,409 copying lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,411 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,414 copying lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,416 copying lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,418 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,420 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,423 copying lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,425 copying lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,427 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,430 copying lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,432 copying lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,434 copying lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,436 copying lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,439 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,441 copying lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,443 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,446 copying lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,448 copying lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,451 copying lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,453 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,455 copying lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,457 copying lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,459 copying lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,462 copying lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,464 copying lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,466 copying lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,469 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,471 copying lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,473 copying lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,475 copying lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,478 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,480 copying lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,482 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,485 copying lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,487 copying lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,489 copying lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,491 copying lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,493 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,496 copying lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,498 copying lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,500 copying lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,503 copying lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,505 copying lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,507 copying lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,509 copying lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,511 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,513 copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,515 copying lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,517 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,519 copying lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,521 copying lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,524 copying lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,526 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,528 copying lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,530 copying lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,532 copying lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,534 copying lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,536 copying lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,538 copying lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,540 copying lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,543 copying lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,545 copying lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,547 copying lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,549 copying lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,551 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,554 copying lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,556 copying lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,558 copying lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,560 copying lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,562 copying lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,564 copying lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,566 copying lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,568 copying lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,570 copying lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,572 copying lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,574 copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,577 copying lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,579 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,581 copying lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,583 copying lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,585 copying lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,587 copying lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,589 copying lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,591 copying lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,593 copying lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,596 copying lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,598 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,600 copying lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,602 copying lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,604 copying lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,607 copying lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:19,609 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,611 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,613 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,615 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,617 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,619 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,621 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,623 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,625 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,627 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,629 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,631 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,633 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,636 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,638 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,640 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,642 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,644 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,646 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,648 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,650 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,653 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,655 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,657 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,659 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,661 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,663 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,665 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,667 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,669 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,671 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,673 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,675 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,677 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,679 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,681 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,683 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,685 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,687 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,689 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,691 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,693 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,695 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,697 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,700 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,702 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,703 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,705 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,707 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:19,709 copying lm_eval/tasks/wikitext/wikitext.yaml -> build/lib/lm_eval/tasks/wikitext 2026-05-11T15:52:19,711 creating build/lib/lm_eval/tasks/arc 2026-05-11T15:52:19,712 copying lm_eval/tasks/arc/arc_easy.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T15:52:19,714 copying lm_eval/tasks/arc/test_arc.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T15:52:19,716 copying lm_eval/tasks/arc/arc_challenge_chat.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T15:52:19,718 copying lm_eval/tasks/arc/arc_challenge.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T15:52:19,720 copying lm_eval/tasks/arc/groups.yaml -> build/lib/lm_eval/tasks/arc 2026-05-11T15:52:19,722 creating build/lib/lm_eval/tasks/swag 2026-05-11T15:52:19,723 copying lm_eval/tasks/swag/swag.yaml -> build/lib/lm_eval/tasks/swag 2026-05-11T15:52:19,725 copying lm_eval/tasks/med_text_classification/med_text_classification_easy.yaml -> build/lib/lm_eval/tasks/med_text_classification 2026-05-11T15:52:19,727 copying lm_eval/tasks/med_text_classification/med_text_classification_hard.yaml -> build/lib/lm_eval/tasks/med_text_classification 2026-05-11T15:52:19,730 creating build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,731 copying lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,733 copying lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,735 copying lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,737 copying lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,739 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,741 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,744 copying lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,746 copying lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,749 copying lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,751 copying lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,753 copying lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,755 copying lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,757 copying lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,760 copying lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,762 copying lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,764 copying lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,766 copying lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,768 copying lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,770 copying lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,772 copying lm_eval/tasks/bbh/fewshot/_bbh_fewshot.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,774 copying lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,776 copying lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,778 copying lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,780 copying lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,782 copying lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,785 copying lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,787 copying lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,789 copying lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:19,791 creating build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,792 copying lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,794 copying lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,797 copying lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,799 copying lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,802 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,804 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,806 copying lm_eval/tasks/bbh/cot_fewshot/_bbh_cot_fewshot.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,808 copying lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,811 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,813 copying lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,815 copying lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,818 copying lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,820 copying lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,822 copying lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,824 copying lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,827 copying lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,829 copying lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,831 copying lm_eval/tasks/bbh/cot_fewshot/_bbh.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,834 copying lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,836 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,838 copying lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,840 copying lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,842 copying lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,844 copying lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,846 copying lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,849 copying lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,851 copying lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,853 copying lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,855 copying lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:19,857 copying lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,860 copying lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,862 copying lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,864 copying lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,866 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,868 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,870 copying lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,872 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,874 copying lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,876 copying lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,878 copying lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,880 copying lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,882 copying lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,884 copying lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,886 copying lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,888 copying lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,890 copying lm_eval/tasks/bbh/cot_zeroshot/_bbh_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,892 copying lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,895 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,897 copying lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,899 copying lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,901 copying lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,903 copying lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,905 copying lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,907 copying lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,909 copying lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,911 copying lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,912 copying lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:19,914 copying lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,916 copying lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,919 copying lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,921 copying lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,923 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,925 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,927 copying lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,929 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,931 copying lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,933 copying lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,935 copying lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,937 copying lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,939 copying lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,941 copying lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,943 copying lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,946 copying lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,948 copying lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,950 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,952 copying lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,954 copying lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,956 copying lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,958 copying lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,960 copying lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,962 copying lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,964 copying lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,966 copying lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,968 copying lm_eval/tasks/bbh/zeroshot/_bbh_zeroshot.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,971 copying lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:19,973 copying lm_eval/tasks/csatqa/csatqa_li.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,975 copying lm_eval/tasks/csatqa/_csatqa.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,977 copying lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,979 copying lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,981 copying lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,983 copying lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,985 copying lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:19,987 copying lm_eval/tasks/xnli/xnli_bg.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:19,989 copying lm_eval/tasks/xnli/xnli_vi.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:19,991 copying lm_eval/tasks/xnli/xnli_en.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:19,993 copying lm_eval/tasks/xnli/xnli_fr.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:19,995 copying lm_eval/tasks/xnli/xnli_zh.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:19,997 copying lm_eval/tasks/xnli/xnli_tr.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:19,999 copying lm_eval/tasks/xnli/xnli_ru.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,001 copying lm_eval/tasks/xnli/xnli_sw.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,004 copying lm_eval/tasks/xnli/xnli_ar.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,005 copying lm_eval/tasks/xnli/xnli_th.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,008 copying lm_eval/tasks/xnli/_xnli.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,010 copying lm_eval/tasks/xnli/xnli_hi.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,012 copying lm_eval/tasks/xnli/xnli_de.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,014 copying lm_eval/tasks/xnli/xnli_ur.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,016 copying lm_eval/tasks/xnli/xnli_es.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,018 copying lm_eval/tasks/xnli/xnli_el.yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:20,020 copying lm_eval/tasks/global_piqa/completions/nag_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,022 copying lm_eval/tasks/global_piqa/completions/por_latn_braz.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,025 copying lm_eval/tasks/global_piqa/completions/zul_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,027 copying lm_eval/tasks/global_piqa/completions/hin_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,029 copying lm_eval/tasks/global_piqa/completions/hye_armn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,031 copying lm_eval/tasks/global_piqa/completions/vie_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,033 copying lm_eval/tasks/global_piqa/completions/bho_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,035 copying lm_eval/tasks/global_piqa/completions/fin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,038 copying lm_eval/tasks/global_piqa/completions/bam_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,040 copying lm_eval/tasks/global_piqa/completions/urh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,042 copying lm_eval/tasks/global_piqa/completions/rwr_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,043 copying lm_eval/tasks/global_piqa/completions/spa_latn_mexi.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,045 copying lm_eval/tasks/global_piqa/completions/acq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,047 copying lm_eval/tasks/global_piqa/completions/ita_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,049 copying lm_eval/tasks/global_piqa/completions/ekk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,052 copying lm_eval/tasks/global_piqa/completions/cat_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,054 copying lm_eval/tasks/global_piqa/completions/isl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,056 copying lm_eval/tasks/global_piqa/completions/ckm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,058 copying lm_eval/tasks/global_piqa/completions/yor_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,060 copying lm_eval/tasks/global_piqa/completions/uzn_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,062 copying lm_eval/tasks/global_piqa/completions/hun_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,064 copying lm_eval/tasks/global_piqa/completions/pol_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,066 copying lm_eval/tasks/global_piqa/completions/mni_mtei.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,068 copying lm_eval/tasks/global_piqa/completions/slk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,070 copying lm_eval/tasks/global_piqa/completions/glg_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,072 copying lm_eval/tasks/global_piqa/completions/bel_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,074 copying lm_eval/tasks/global_piqa/completions/tel_telu.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,076 copying lm_eval/tasks/global_piqa/completions/swe_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,078 copying lm_eval/tasks/global_piqa/completions/yue_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,081 copying lm_eval/tasks/global_piqa/completions/jpn_jpan.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,083 copying lm_eval/tasks/global_piqa/completions/apc_arab_pale.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,085 copying lm_eval/tasks/global_piqa/completions/snd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,087 copying lm_eval/tasks/global_piqa/completions/slv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,089 copying lm_eval/tasks/global_piqa/completions/dhd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,091 copying lm_eval/tasks/global_piqa/completions/bos_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,094 copying lm_eval/tasks/global_piqa/completions/slv_latn_cerk.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,096 copying lm_eval/tasks/global_piqa/completions/ukr_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,098 copying lm_eval/tasks/global_piqa/completions/guj_gujr.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,100 copying lm_eval/tasks/global_piqa/completions/tam_taml.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,102 copying lm_eval/tasks/global_piqa/completions/fra_latn_fran.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,104 copying lm_eval/tasks/global_piqa/completions/mni_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,106 copying lm_eval/tasks/global_piqa/completions/lit_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,108 copying lm_eval/tasks/global_piqa/completions/jav_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,110 copying lm_eval/tasks/global_piqa/completions/_global_piqa.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,113 copying lm_eval/tasks/global_piqa/completions/tha_thai.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,115 copying lm_eval/tasks/global_piqa/completions/srp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,117 copying lm_eval/tasks/global_piqa/completions/apc_arab_leba.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,120 copying lm_eval/tasks/global_piqa/completions/spa_latn_peru.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,122 copying lm_eval/tasks/global_piqa/completions/ind_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,124 copying lm_eval/tasks/global_piqa/completions/ben_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,126 copying lm_eval/tasks/global_piqa/completions/cmn_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,128 copying lm_eval/tasks/global_piqa/completions/ell_grek.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,130 copying lm_eval/tasks/global_piqa/completions/ekp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,133 copying lm_eval/tasks/global_piqa/completions/srp_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,135 copying lm_eval/tasks/global_piqa/completions/tur_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,137 copying lm_eval/tasks/global_piqa/completions/uig_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,139 copying lm_eval/tasks/global_piqa/completions/tgl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,141 copying lm_eval/tasks/global_piqa/completions/arq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,143 copying lm_eval/tasks/global_piqa/completions/cmn_hans.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,146 copying lm_eval/tasks/global_piqa/completions/swh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,148 copying lm_eval/tasks/global_piqa/completions/arb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,150 copying lm_eval/tasks/global_piqa/completions/mal_mlym.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,152 copying lm_eval/tasks/global_piqa/completions/rus_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,154 copying lm_eval/tasks/global_piqa/completions/arz_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,157 copying lm_eval/tasks/global_piqa/completions/por_latn_port.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,159 copying lm_eval/tasks/global_piqa/completions/aeb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,161 copying lm_eval/tasks/global_piqa/completions/nob_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,163 copying lm_eval/tasks/global_piqa/completions/snd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,165 copying lm_eval/tasks/global_piqa/completions/kan_knda.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,167 copying lm_eval/tasks/global_piqa/completions/nno_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,169 copying lm_eval/tasks/global_piqa/completions/ben_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,172 copying lm_eval/tasks/global_piqa/completions/ron_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,174 copying lm_eval/tasks/global_piqa/completions/amh_ethi.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,176 copying lm_eval/tasks/global_piqa/completions/iso_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,178 copying lm_eval/tasks/global_piqa/completions/als_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,181 copying lm_eval/tasks/global_piqa/completions/ckb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,183 copying lm_eval/tasks/global_piqa/completions/kir_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,185 copying lm_eval/tasks/global_piqa/completions/kaz_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,187 copying lm_eval/tasks/global_piqa/completions/npi_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,189 copying lm_eval/tasks/global_piqa/completions/fao_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,192 copying lm_eval/tasks/global_piqa/completions/luo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,194 copying lm_eval/tasks/global_piqa/completions/bsk_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,196 copying lm_eval/tasks/global_piqa/completions/azj_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,198 copying lm_eval/tasks/global_piqa/completions/acm_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,201 copying lm_eval/tasks/global_piqa/completions/hau_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,203 copying lm_eval/tasks/global_piqa/completions/idu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,206 copying lm_eval/tasks/global_piqa/completions/lin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,208 copying lm_eval/tasks/global_piqa/completions/afb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,210 copying lm_eval/tasks/global_piqa/completions/haw_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,213 copying lm_eval/tasks/global_piqa/completions/ces_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,215 copying lm_eval/tasks/global_piqa/completions/kor_hang.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,217 copying lm_eval/tasks/global_piqa/completions/slk_latn_sari.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,220 copying lm_eval/tasks/global_piqa/completions/fra_latn_cana.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,222 copying lm_eval/tasks/global_piqa/completions/kat_geor.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,224 copying lm_eval/tasks/global_piqa/completions/mar_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,227 copying lm_eval/tasks/global_piqa/completions/kin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,229 copying lm_eval/tasks/global_piqa/completions/urd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,231 copying lm_eval/tasks/global_piqa/completions/nld_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,233 copying lm_eval/tasks/global_piqa/completions/deu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,235 copying lm_eval/tasks/global_piqa/completions/pcm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,237 copying lm_eval/tasks/global_piqa/completions/eng_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,239 copying lm_eval/tasks/global_piqa/completions/urd_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,241 copying lm_eval/tasks/global_piqa/completions/ary_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,243 copying lm_eval/tasks/global_piqa/completions/mkd_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,246 copying lm_eval/tasks/global_piqa/completions/pan_guru.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,248 copying lm_eval/tasks/global_piqa/completions/heb_hebr.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,250 copying lm_eval/tasks/global_piqa/completions/sin_sinh.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,252 copying lm_eval/tasks/global_piqa/completions/ars_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,254 copying lm_eval/tasks/global_piqa/completions/spa_latn_spai.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,256 copying lm_eval/tasks/global_piqa/completions/asm_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,258 copying lm_eval/tasks/global_piqa/completions/ibo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,260 copying lm_eval/tasks/global_piqa/completions/bul_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,263 copying lm_eval/tasks/global_piqa/completions/apc_arab_jord.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,265 copying lm_eval/tasks/global_piqa/completions/hrv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,267 copying lm_eval/tasks/global_piqa/completions/zsm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,269 copying lm_eval/tasks/global_piqa/completions/pes_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,272 copying lm_eval/tasks/global_piqa/completions/apc_arab_syri.yaml -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:20,274 copying lm_eval/tasks/global_piqa/prompted/nag_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,276 copying lm_eval/tasks/global_piqa/prompted/por_latn_braz.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,278 copying lm_eval/tasks/global_piqa/prompted/zul_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,280 copying lm_eval/tasks/global_piqa/prompted/hin_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,283 copying lm_eval/tasks/global_piqa/prompted/hye_armn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,285 copying lm_eval/tasks/global_piqa/prompted/vie_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,287 copying lm_eval/tasks/global_piqa/prompted/bho_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,289 copying lm_eval/tasks/global_piqa/prompted/fin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,292 copying lm_eval/tasks/global_piqa/prompted/bam_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,294 copying lm_eval/tasks/global_piqa/prompted/urh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,296 copying lm_eval/tasks/global_piqa/prompted/rwr_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,298 copying lm_eval/tasks/global_piqa/prompted/spa_latn_mexi.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,300 copying lm_eval/tasks/global_piqa/prompted/acq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,302 copying lm_eval/tasks/global_piqa/prompted/ita_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,304 copying lm_eval/tasks/global_piqa/prompted/ekk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,306 copying lm_eval/tasks/global_piqa/prompted/cat_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,308 copying lm_eval/tasks/global_piqa/prompted/isl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,311 copying lm_eval/tasks/global_piqa/prompted/ckm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,313 copying lm_eval/tasks/global_piqa/prompted/yor_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,315 copying lm_eval/tasks/global_piqa/prompted/uzn_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,317 copying lm_eval/tasks/global_piqa/prompted/hun_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,319 copying lm_eval/tasks/global_piqa/prompted/pol_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,321 copying lm_eval/tasks/global_piqa/prompted/mni_mtei.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,323 copying lm_eval/tasks/global_piqa/prompted/slk_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,325 copying lm_eval/tasks/global_piqa/prompted/glg_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,327 copying lm_eval/tasks/global_piqa/prompted/bel_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,330 copying lm_eval/tasks/global_piqa/prompted/tel_telu.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,332 copying lm_eval/tasks/global_piqa/prompted/swe_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,334 copying lm_eval/tasks/global_piqa/prompted/yue_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,336 copying lm_eval/tasks/global_piqa/prompted/jpn_jpan.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,339 copying lm_eval/tasks/global_piqa/prompted/apc_arab_pale.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,341 copying lm_eval/tasks/global_piqa/prompted/snd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,343 copying lm_eval/tasks/global_piqa/prompted/slv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,345 copying lm_eval/tasks/global_piqa/prompted/dhd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,347 copying lm_eval/tasks/global_piqa/prompted/bos_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,350 copying lm_eval/tasks/global_piqa/prompted/slv_latn_cerk.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,352 copying lm_eval/tasks/global_piqa/prompted/ukr_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,354 copying lm_eval/tasks/global_piqa/prompted/guj_gujr.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,356 copying lm_eval/tasks/global_piqa/prompted/tam_taml.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,359 copying lm_eval/tasks/global_piqa/prompted/fra_latn_fran.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,361 copying lm_eval/tasks/global_piqa/prompted/mni_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,363 copying lm_eval/tasks/global_piqa/prompted/lit_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,366 copying lm_eval/tasks/global_piqa/prompted/jav_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,368 copying lm_eval/tasks/global_piqa/prompted/_global_piqa.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,371 copying lm_eval/tasks/global_piqa/prompted/tha_thai.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,373 copying lm_eval/tasks/global_piqa/prompted/srp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,375 copying lm_eval/tasks/global_piqa/prompted/apc_arab_leba.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,377 copying lm_eval/tasks/global_piqa/prompted/spa_latn_peru.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,380 copying lm_eval/tasks/global_piqa/prompted/ind_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,382 copying lm_eval/tasks/global_piqa/prompted/ben_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,385 copying lm_eval/tasks/global_piqa/prompted/cmn_hant.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,388 copying lm_eval/tasks/global_piqa/prompted/ell_grek.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,390 copying lm_eval/tasks/global_piqa/prompted/ekp_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,392 copying lm_eval/tasks/global_piqa/prompted/srp_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,394 copying lm_eval/tasks/global_piqa/prompted/tur_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,396 copying lm_eval/tasks/global_piqa/prompted/uig_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,398 copying lm_eval/tasks/global_piqa/prompted/tgl_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,401 copying lm_eval/tasks/global_piqa/prompted/arq_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,403 copying lm_eval/tasks/global_piqa/prompted/cmn_hans.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,405 copying lm_eval/tasks/global_piqa/prompted/swh_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,407 copying lm_eval/tasks/global_piqa/prompted/arb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,410 copying lm_eval/tasks/global_piqa/prompted/mal_mlym.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,412 copying lm_eval/tasks/global_piqa/prompted/rus_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,415 copying lm_eval/tasks/global_piqa/prompted/arz_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,418 copying lm_eval/tasks/global_piqa/prompted/por_latn_port.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,421 copying lm_eval/tasks/global_piqa/prompted/aeb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,424 copying lm_eval/tasks/global_piqa/prompted/nob_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,427 copying lm_eval/tasks/global_piqa/prompted/snd_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,431 copying lm_eval/tasks/global_piqa/prompted/kan_knda.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,434 copying lm_eval/tasks/global_piqa/prompted/nno_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,437 copying lm_eval/tasks/global_piqa/prompted/ben_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,440 copying lm_eval/tasks/global_piqa/prompted/ron_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,443 copying lm_eval/tasks/global_piqa/prompted/amh_ethi.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,446 copying lm_eval/tasks/global_piqa/prompted/iso_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,449 copying lm_eval/tasks/global_piqa/prompted/als_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,452 copying lm_eval/tasks/global_piqa/prompted/ckb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,455 copying lm_eval/tasks/global_piqa/prompted/kir_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,458 copying lm_eval/tasks/global_piqa/prompted/kaz_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,461 copying lm_eval/tasks/global_piqa/prompted/npi_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,464 copying lm_eval/tasks/global_piqa/prompted/fao_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,467 copying lm_eval/tasks/global_piqa/prompted/luo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,471 copying lm_eval/tasks/global_piqa/prompted/bsk_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,475 copying lm_eval/tasks/global_piqa/prompted/azj_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,478 copying lm_eval/tasks/global_piqa/prompted/acm_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,481 copying lm_eval/tasks/global_piqa/prompted/hau_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,483 copying lm_eval/tasks/global_piqa/prompted/idu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,486 copying lm_eval/tasks/global_piqa/prompted/lin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,489 copying lm_eval/tasks/global_piqa/prompted/afb_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,492 copying lm_eval/tasks/global_piqa/prompted/haw_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,494 copying lm_eval/tasks/global_piqa/prompted/ces_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,497 copying lm_eval/tasks/global_piqa/prompted/kor_hang.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,500 copying lm_eval/tasks/global_piqa/prompted/slk_latn_sari.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,502 copying lm_eval/tasks/global_piqa/prompted/fra_latn_cana.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,506 copying lm_eval/tasks/global_piqa/prompted/kat_geor.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,509 copying lm_eval/tasks/global_piqa/prompted/mar_deva.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,512 copying lm_eval/tasks/global_piqa/prompted/kin_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,515 copying lm_eval/tasks/global_piqa/prompted/urd_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,517 copying lm_eval/tasks/global_piqa/prompted/nld_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,520 copying lm_eval/tasks/global_piqa/prompted/deu_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,523 copying lm_eval/tasks/global_piqa/prompted/pcm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,526 copying lm_eval/tasks/global_piqa/prompted/eng_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,529 copying lm_eval/tasks/global_piqa/prompted/urd_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,532 copying lm_eval/tasks/global_piqa/prompted/ary_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,535 copying lm_eval/tasks/global_piqa/prompted/mkd_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,537 copying lm_eval/tasks/global_piqa/prompted/pan_guru.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,540 copying lm_eval/tasks/global_piqa/prompted/heb_hebr.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,543 copying lm_eval/tasks/global_piqa/prompted/sin_sinh.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,546 copying lm_eval/tasks/global_piqa/prompted/ars_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,549 copying lm_eval/tasks/global_piqa/prompted/spa_latn_spai.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,552 copying lm_eval/tasks/global_piqa/prompted/asm_beng.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,555 copying lm_eval/tasks/global_piqa/prompted/ibo_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,558 copying lm_eval/tasks/global_piqa/prompted/bul_cyrl.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,560 copying lm_eval/tasks/global_piqa/prompted/apc_arab_jord.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,563 copying lm_eval/tasks/global_piqa/prompted/hrv_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,566 copying lm_eval/tasks/global_piqa/prompted/zsm_latn.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,568 copying lm_eval/tasks/global_piqa/prompted/pes_arab.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,570 copying lm_eval/tasks/global_piqa/prompted/apc_arab_syri.yaml -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:20,572 creating build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,573 copying lm_eval/tasks/bhs/basque-S-S_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,576 copying lm_eval/tasks/bhs/hindi-S_PossPRN_PossN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,578 copying lm_eval/tasks/bhs/bhs_hindi.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,580 copying lm_eval/tasks/bhs/bhs_swahili.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,582 copying lm_eval/tasks/bhs/basque-S-S_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,584 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_ni_AN.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,586 copying lm_eval/tasks/bhs/hindi-S_ne_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,588 copying lm_eval/tasks/bhs/swahili-N_of_Poss_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,591 copying lm_eval/tasks/bhs/basque-S-IO_S_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,593 copying lm_eval/tasks/bhs/basque-IO-IO_S_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,595 copying lm_eval/tasks/bhs/hindi-S_ne_PossPRN_PossN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,597 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,600 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,603 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_V_ni_AN.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,606 copying lm_eval/tasks/bhs/swahili-N_of_Poss_ni_A.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,609 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_ni_A.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,612 copying lm_eval/tasks/bhs/basque-DO-S_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,615 copying lm_eval/tasks/bhs/basque-DO-S_IO_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,618 copying lm_eval/tasks/bhs/basque-IO-S_IO_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,621 copying lm_eval/tasks/bhs/bhs_basque.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,625 copying lm_eval/tasks/bhs/hindi-S_ne_PossPRN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,628 copying lm_eval/tasks/bhs/hindi-S_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,631 copying lm_eval/tasks/bhs/basque-S-S_IO_DO_V_AUX.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,634 copying lm_eval/tasks/bhs/hindi-S_PossPRN_O_V.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,637 copying lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V1_V2.yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:20,640 creating build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T15:52:20,641 copying lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T15:52:20,644 copying lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T15:52:20,647 copying lm_eval/tasks/eus_reading/eus_reading.yaml -> build/lib/lm_eval/tasks/eus_reading 2026-05-11T15:52:20,650 copying lm_eval/tasks/longbench/trec_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,653 copying lm_eval/tasks/longbench/multi_news.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,655 copying lm_eval/tasks/longbench/_longbench_code.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,657 copying lm_eval/tasks/longbench/repobench-p.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,659 copying lm_eval/tasks/longbench/musique.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,662 copying lm_eval/tasks/longbench/_longbench_multi.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,664 copying lm_eval/tasks/longbench/hotpotqa_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,667 copying lm_eval/tasks/longbench/passage_count_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,671 copying lm_eval/tasks/longbench/narrativeqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,673 copying lm_eval/tasks/longbench/lcc.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,675 copying lm_eval/tasks/longbench/samsum.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,678 copying lm_eval/tasks/longbench/gov_report.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,682 copying lm_eval/tasks/longbench/multi_news_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,685 copying lm_eval/tasks/longbench/_longbench_fewshot.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,688 copying lm_eval/tasks/longbench/multifieldqa_zh.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,691 copying lm_eval/tasks/longbench/_longbench_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,694 copying lm_eval/tasks/longbench/_longbench_synthetic_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,697 copying lm_eval/tasks/longbench/_longbench_fewshot_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,700 copying lm_eval/tasks/longbench/_longbench_single.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,702 copying lm_eval/tasks/longbench/_longbench_multi_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,704 copying lm_eval/tasks/longbench/triviaqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,707 copying lm_eval/tasks/longbench/hotpotqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,709 copying lm_eval/tasks/longbench/qmsum.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,711 copying lm_eval/tasks/longbench/passage_count.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,714 copying lm_eval/tasks/longbench/samsum_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,716 copying lm_eval/tasks/longbench/vcsum.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,718 copying lm_eval/tasks/longbench/_longbench_single_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,721 copying lm_eval/tasks/longbench/qasper_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,723 copying lm_eval/tasks/longbench/passage_retrieval_en.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,725 copying lm_eval/tasks/longbench/passage_retrieval_zh.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,727 copying lm_eval/tasks/longbench/2wikimqa_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,729 copying lm_eval/tasks/longbench/triviaqa_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,731 copying lm_eval/tasks/longbench/_longbench_summarization_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,734 copying lm_eval/tasks/longbench/dureader.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,736 copying lm_eval/tasks/longbench/_longbench_code_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,738 copying lm_eval/tasks/longbench/qasper.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,740 copying lm_eval/tasks/longbench/_longbench_synthetic.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,742 copying lm_eval/tasks/longbench/trec.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,744 copying lm_eval/tasks/longbench/repobench-p_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,746 copying lm_eval/tasks/longbench/_longbench_summarization.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,748 copying lm_eval/tasks/longbench/passage_retrieval_en_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,751 copying lm_eval/tasks/longbench/gov_report_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,753 copying lm_eval/tasks/longbench/2wikimqa.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,755 copying lm_eval/tasks/longbench/lcc_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,757 copying lm_eval/tasks/longbench/_longbench.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,759 copying lm_eval/tasks/longbench/multifieldqa_en_e.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,761 copying lm_eval/tasks/longbench/lsht.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,764 copying lm_eval/tasks/longbench/multifieldqa_en.yaml -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:20,766 creating build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,767 copying lm_eval/tasks/mmlu/continuation/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,769 copying lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,771 copying lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,774 copying lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,776 copying lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,778 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,780 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,782 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,785 copying lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,787 copying lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,790 copying lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,792 copying lm_eval/tasks/mmlu/continuation/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,795 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,798 copying lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,800 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,803 copying lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,805 copying lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,807 copying lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,809 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,812 copying lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,814 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,816 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,818 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,821 copying lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,823 copying lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,825 copying lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,827 copying lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,830 copying lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,832 copying lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,834 copying lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,836 copying lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,839 copying lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,841 copying lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,844 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,847 copying lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,849 copying lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,851 copying lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,853 copying lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,856 copying lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,858 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,860 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,862 copying lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,865 copying lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,868 copying lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,870 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,873 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,875 copying lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,877 copying lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,879 copying lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,881 copying lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,884 copying lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,886 copying lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,888 copying lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,890 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,892 copying lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,894 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,897 copying lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,899 copying lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:20,901 creating build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,902 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,905 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,907 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,910 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,912 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,915 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,917 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,920 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,922 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,925 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,928 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,931 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,933 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,936 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,939 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,941 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,943 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,945 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,948 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,950 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,953 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,956 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,958 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,961 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,963 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,966 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,968 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,970 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,973 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,975 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,978 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,980 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,982 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,985 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,987 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,990 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,992 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,994 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:20,997 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,000 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,003 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,005 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,008 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,010 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,013 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,015 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,018 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,020 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,023 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,025 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,028 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,030 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,033 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,035 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,038 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,040 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,043 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,045 copying lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:21,048 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,050 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,052 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,055 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,057 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,059 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,061 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,064 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,066 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,068 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,071 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,073 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,075 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,077 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,079 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,081 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,083 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,085 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,087 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,089 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,091 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,093 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,095 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,097 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,099 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,102 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,104 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,106 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,108 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,111 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,113 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,115 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,118 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,120 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,122 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,125 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,127 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,129 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,132 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,134 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,136 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,138 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,141 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,143 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,145 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,147 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,149 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,151 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,153 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,156 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,158 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,160 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,162 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,164 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,167 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,169 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,171 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,173 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:21,176 creating build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,177 copying lm_eval/tasks/mmlu/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,179 copying lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,181 copying lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,184 copying lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,186 copying lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,188 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,190 copying lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,192 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,195 copying lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,197 copying lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,199 copying lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,201 copying lm_eval/tasks/mmlu/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,203 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,205 copying lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,208 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,210 copying lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,213 copying lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,215 copying lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,217 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,219 copying lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,221 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,223 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,226 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,228 copying lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,230 copying lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,232 copying lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,234 copying lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,237 copying lm_eval/tasks/mmlu/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,239 copying lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,241 copying lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,243 copying lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,246 copying lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,248 copying lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,250 copying lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,252 copying lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,255 copying lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,257 copying lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,259 copying lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,262 copying lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,264 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,266 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,268 copying lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,270 copying lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,272 copying lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,274 copying lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,276 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,279 copying lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,281 copying lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,283 copying lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,285 copying lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,287 copying lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,290 copying lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,292 copying lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,294 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,296 copying lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,298 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,301 copying lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,303 copying lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:21,305 creating build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,306 copying lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,309 copying lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,311 copying lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,313 copying lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,315 copying lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,318 copying lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,320 copying lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,322 copying lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,324 copying lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,327 copying lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,329 copying lm_eval/tasks/mmlu/default/_mmlu_social_sciences.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,331 copying lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,333 copying lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,335 copying lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,337 copying lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,340 copying lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,342 copying lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,344 copying lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,346 copying lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,348 copying lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,350 copying lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,352 copying lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,355 copying lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,357 copying lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,359 copying lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,362 copying lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,364 copying lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,366 copying lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,368 copying lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,371 copying lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,373 copying lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,375 copying lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,377 copying lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,380 copying lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,382 copying lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,384 copying lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,386 copying lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,389 copying lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,391 copying lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,393 copying lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,396 copying lm_eval/tasks/mmlu/default/_mmlu_stem.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,398 copying lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,400 copying lm_eval/tasks/mmlu/default/_mmlu_humanities.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,402 copying lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,405 copying lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,407 copying lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,409 copying lm_eval/tasks/mmlu/default/_mmlu_other.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,411 copying lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,413 copying lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,415 copying lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,417 copying lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,419 copying lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,422 copying lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,424 copying lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,426 copying lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,428 copying lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,431 copying lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,433 copying lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,435 copying lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,437 copying lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,440 copying lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,442 copying lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:21,444 copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,446 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,449 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,451 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,453 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,455 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,457 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,460 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,462 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,464 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,467 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,469 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,471 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,473 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,475 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,477 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,479 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,482 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,484 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,486 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,488 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,490 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,492 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,495 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,497 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,499 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,501 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,503 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,506 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,508 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,510 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,512 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,514 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,516 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,519 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,521 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,523 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,526 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,528 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,530 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,532 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,535 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,537 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,539 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,541 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,543 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,546 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,548 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,550 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,552 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,554 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,557 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,559 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,561 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,563 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,566 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,568 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,570 copying lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:21,572 creating build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,573 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,576 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,578 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,580 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,583 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,585 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,588 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,590 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,593 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,595 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,597 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,599 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,601 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,603 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,606 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,608 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,610 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,613 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,615 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,617 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,620 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,622 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,625 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,627 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,630 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,633 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,635 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,637 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,640 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,642 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,644 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,646 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,648 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,651 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,653 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,655 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,657 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,659 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,661 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,664 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,666 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,669 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,672 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,675 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,677 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,680 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,682 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,684 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,686 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,689 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,691 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,693 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,696 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,698 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,700 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,702 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,704 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,706 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:21,708 creating build/lib/lm_eval/tasks/headqa 2026-05-11T15:52:21,709 copying lm_eval/tasks/headqa/headqa_es.yaml -> build/lib/lm_eval/tasks/headqa 2026-05-11T15:52:21,711 copying lm_eval/tasks/headqa/headqa_en.yaml -> build/lib/lm_eval/tasks/headqa 2026-05-11T15:52:21,714 copying lm_eval/tasks/blimp/intransitive.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,716 copying lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,718 copying lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,720 copying lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,722 copying lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,724 copying lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,727 copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,729 copying lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,731 copying lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,733 copying lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,735 copying lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,738 copying lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,740 copying lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,742 copying lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,744 copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,746 copying lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,749 copying lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,751 copying lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,753 copying lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,756 copying lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,758 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,760 copying lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,762 copying lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,765 copying lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,767 copying lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,769 copying lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,771 copying lm_eval/tasks/blimp/drop_argument.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,773 copying lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,775 copying lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,777 copying lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,779 copying lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,781 copying lm_eval/tasks/blimp/inchoative.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,783 copying lm_eval/tasks/blimp/causative.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,785 copying lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,787 copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,789 copying lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,791 copying lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,793 copying lm_eval/tasks/blimp/passive_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,795 copying lm_eval/tasks/blimp/adjunct_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,797 copying lm_eval/tasks/blimp/_blimp.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,799 copying lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,801 copying lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,803 copying lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,805 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,807 copying lm_eval/tasks/blimp/wh_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,809 copying lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,811 copying lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,813 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,815 copying lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,817 copying lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,819 copying lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,821 copying lm_eval/tasks/blimp/complex_NP_island.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,823 copying lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,825 copying lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,827 copying lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,829 copying lm_eval/tasks/blimp/npi_present_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,832 copying lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,834 copying lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,836 copying lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,838 copying lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,840 copying lm_eval/tasks/blimp/passive_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,842 copying lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,844 copying lm_eval/tasks/blimp/transitive.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,846 copying lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,847 copying lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,849 copying lm_eval/tasks/blimp/only_npi_scope.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,851 copying lm_eval/tasks/blimp/npi_present_2.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,853 copying lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:21,855 creating build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,856 copying lm_eval/tasks/mela/mela_en.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,858 copying lm_eval/tasks/mela/mela_fr.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,861 copying lm_eval/tasks/mela/mela_it.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,863 copying lm_eval/tasks/mela/mela_ru.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,865 copying lm_eval/tasks/mela/mela_es.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,867 copying lm_eval/tasks/mela/mela_de.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,869 copying lm_eval/tasks/mela/mela_ar.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,871 copying lm_eval/tasks/mela/mela_ja.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,873 copying lm_eval/tasks/mela/_mela.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,875 copying lm_eval/tasks/mela/mela_zh.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,877 copying lm_eval/tasks/mela/mela_is.yaml -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:21,879 copying lm_eval/tasks/mmmu/mmmu_mechanical_engineering.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,881 copying lm_eval/tasks/mmmu/mmmu_basic_medical_science.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,883 copying lm_eval/tasks/mmmu/mmmu_accounting.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,885 copying lm_eval/tasks/mmmu/mmmu_design.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,887 copying lm_eval/tasks/mmmu/mmmu_sociology.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,889 copying lm_eval/tasks/mmmu/_health_and_medicine.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,891 copying lm_eval/tasks/mmmu/mmmu_public_health.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,893 copying lm_eval/tasks/mmmu/mmmu_physics.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,895 copying lm_eval/tasks/mmmu/_mmmu.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,897 copying lm_eval/tasks/mmmu/mmmu_manage.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,899 copying lm_eval/tasks/mmmu/mmmu_chemistry.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,901 copying lm_eval/tasks/mmmu/_art_and_design.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,903 copying lm_eval/tasks/mmmu/mmmu_computer_science.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,905 copying lm_eval/tasks/mmmu/mmmu_music.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,907 copying lm_eval/tasks/mmmu/mmmu_finance.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,910 copying lm_eval/tasks/mmmu/mmmu_agriculture.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,912 copying lm_eval/tasks/mmmu/mmmu_materials.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,914 copying lm_eval/tasks/mmmu/mmmu_marketing.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,916 copying lm_eval/tasks/mmmu/_business.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,918 copying lm_eval/tasks/mmmu/mmmu_psychology.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,920 copying lm_eval/tasks/mmmu/mmmu_energy_and_power.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,922 copying lm_eval/tasks/mmmu/mmmu_clinical_medicine.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,924 copying lm_eval/tasks/mmmu/mmmu_economics.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,926 copying lm_eval/tasks/mmmu/mmmu_literature.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,929 copying lm_eval/tasks/mmmu/_science.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,931 copying lm_eval/tasks/mmmu/mmmu_art.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,933 copying lm_eval/tasks/mmmu/_tech_and_engineering.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,935 copying lm_eval/tasks/mmmu/mmmu_diagnostics_and_laboratory_medicine.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,937 copying lm_eval/tasks/mmmu/mmmu_electronics.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,939 copying lm_eval/tasks/mmmu/_humanities_and_social_sciences.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,941 copying lm_eval/tasks/mmmu/mmmu_pharmacy.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,943 copying lm_eval/tasks/mmmu/mmmu_geography.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,945 copying lm_eval/tasks/mmmu/mmmu_biology.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,947 copying lm_eval/tasks/mmmu/mmmu_math.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,950 copying lm_eval/tasks/mmmu/mmmu_history.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,952 copying lm_eval/tasks/mmmu/mmmu_art_theory.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,954 copying lm_eval/tasks/mmmu/mmmu_architecture_and_engineering.yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:21,956 copying lm_eval/tasks/histoires_morales/histoires_morales.yaml -> build/lib/lm_eval/tasks/histoires_morales 2026-05-11T15:52:21,958 creating build/lib/lm_eval/tasks/prost 2026-05-11T15:52:21,959 copying lm_eval/tasks/prost/corypaik_prost.yaml -> build/lib/lm_eval/tasks/prost 2026-05-11T15:52:21,961 creating build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,963 copying lm_eval/tasks/mastermind/mastermind_46_easy.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,965 copying lm_eval/tasks/mastermind/mastermind_46_hard.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,967 copying lm_eval/tasks/mastermind/mastermind_35_easy.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,969 copying lm_eval/tasks/mastermind/mastermind_35_hard.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,971 copying lm_eval/tasks/mastermind/mastermind_24_easy.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,974 copying lm_eval/tasks/mastermind/mastermind_24_hard.yaml -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:21,976 creating build/lib/lm_eval/tasks/anli 2026-05-11T15:52:21,977 copying lm_eval/tasks/anli/anli_r1.yaml -> build/lib/lm_eval/tasks/anli 2026-05-11T15:52:21,979 copying lm_eval/tasks/anli/anli_r2.yaml -> build/lib/lm_eval/tasks/anli 2026-05-11T15:52:21,982 copying lm_eval/tasks/anli/anli_r3.yaml -> build/lib/lm_eval/tasks/anli 2026-05-11T15:52:21,984 creating build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,985 copying lm_eval/tasks/longbench2/_longbench2_single.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,987 copying lm_eval/tasks/longbench2/legal_single.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,990 copying lm_eval/tasks/longbench2/lit_single_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,992 copying lm_eval/tasks/longbench2/many_shot.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,994 copying lm_eval/tasks/longbench2/agent_history.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,997 copying lm_eval/tasks/longbench2/graph.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:21,999 copying lm_eval/tasks/longbench2/_longbench2_structured.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,001 copying lm_eval/tasks/longbench2/table.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,003 copying lm_eval/tasks/longbench2/legal_multi.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,005 copying lm_eval/tasks/longbench2/_longbench2_multi.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,008 copying lm_eval/tasks/longbench2/detective.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,010 copying lm_eval/tasks/longbench2/longbench2_code.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,012 copying lm_eval/tasks/longbench2/academic_single.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,014 copying lm_eval/tasks/longbench2/user_guide.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,016 copying lm_eval/tasks/longbench2/_longbench2_incontext.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,019 copying lm_eval/tasks/longbench2/_longbench2_history.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,021 copying lm_eval/tasks/longbench2/news_multi.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,023 copying lm_eval/tasks/longbench2/govt_multi_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,025 copying lm_eval/tasks/longbench2/_longbench2.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,027 copying lm_eval/tasks/longbench2/dialogue_history.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,030 copying lm_eval/tasks/longbench2/govt_single_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,032 copying lm_eval/tasks/longbench2/academic_multi_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,034 copying lm_eval/tasks/longbench2/translate.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,036 copying lm_eval/tasks/longbench2/fin_single_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,039 copying lm_eval/tasks/longbench2/fin_multi_doc.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,042 copying lm_eval/tasks/longbench2/event_order.yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:22,044 creating build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:22,045 copying lm_eval/tasks/kormedmcqa/pharm.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:22,047 copying lm_eval/tasks/kormedmcqa/doctor.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:22,050 copying lm_eval/tasks/kormedmcqa/nurse.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:22,052 copying lm_eval/tasks/kormedmcqa/_kormedmcqa.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:22,054 copying lm_eval/tasks/kormedmcqa/dentist.yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:22,057 copying lm_eval/tasks/afrimmlu/translate/afrimmlu_tt.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate 2026-05-11T15:52:22,059 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,061 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,064 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,066 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,068 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,071 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,073 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,075 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,077 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,079 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,082 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,084 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,086 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,088 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,090 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,093 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,095 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:22,097 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,100 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,102 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,104 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,106 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,109 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,111 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,113 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,116 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,118 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,120 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,123 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,125 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,127 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,130 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,132 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,134 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:22,136 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,139 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,141 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,144 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,146 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,148 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,150 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,152 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,154 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,156 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,159 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,161 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,164 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,166 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,168 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,170 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,172 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:22,175 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,177 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,179 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,181 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,184 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,186 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,189 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,191 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,193 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,195 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,197 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,200 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,202 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,204 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,206 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,208 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,210 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:22,213 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,215 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,217 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,219 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,222 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,224 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,226 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,228 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,231 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,233 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,235 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,238 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,240 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,242 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,244 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,246 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,248 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:22,251 copying lm_eval/tasks/afrimmlu/direct/afrimmlu.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct 2026-05-11T15:52:22,253 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,255 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,257 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,259 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,262 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,264 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,266 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,268 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,270 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,272 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,275 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,277 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,279 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,281 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,283 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,286 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,288 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,290 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:22,292 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,295 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,297 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,299 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,301 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,304 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,306 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,308 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,310 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,312 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,314 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,316 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,319 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,321 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,323 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,325 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,327 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,330 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:22,332 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,334 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,336 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,338 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,340 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,343 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,345 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,347 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,349 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,351 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,354 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,356 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,358 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,360 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,363 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,365 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,367 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,369 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:22,371 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,374 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,376 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,378 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,380 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,382 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,385 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,387 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,389 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,391 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,394 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,396 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,398 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,401 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,403 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,405 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,407 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,410 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:22,412 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,414 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_kin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,416 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_twi.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,419 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,421 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sna.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,423 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lug.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,426 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_fra.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,428 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_xho.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,430 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_orm.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,433 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_hau.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,435 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sot.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,437 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_yor.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,439 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_zul.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,441 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_swa.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,443 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_eng.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,446 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lin.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,448 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_wol.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,450 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_amh.yaml -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:22,452 copying lm_eval/tasks/mbpp/mbpp_instruct.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:22,454 copying lm_eval/tasks/mbpp/mbpp_plus_instruct.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:22,457 copying lm_eval/tasks/mbpp/mbpp.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:22,459 copying lm_eval/tasks/mbpp/mbpp_plus.yaml -> build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:22,461 creating build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,462 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,464 copying lm_eval/tasks/blimp_nl/determiners__geen_expletive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,467 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,469 copying lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,471 copying lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,473 copying lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,476 copying lm_eval/tasks/blimp_nl/argument_structure__argument_number_in_transitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,478 copying lm_eval/tasks/blimp_nl/complementive__position_verb.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,480 copying lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,482 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,485 copying lm_eval/tasks/blimp_nl/topicalization__island.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,487 copying lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,489 copying lm_eval/tasks/blimp_nl/complementive__position_adverb.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,491 copying lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,494 copying lm_eval/tasks/blimp_nl/auxiliaries__perfect.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,496 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,498 copying lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,500 copying lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,502 copying lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,505 copying lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,507 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,509 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,511 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,513 copying lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,516 copying lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,518 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,520 copying lm_eval/tasks/blimp_nl/adpositional_phrases__argument_r_extraction.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,522 copying lm_eval/tasks/blimp_nl/auxiliaries__order_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,524 copying lm_eval/tasks/blimp_nl/verb_second__order_main.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,527 copying lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,529 copying lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,531 copying lm_eval/tasks/blimp_nl/determiners__geen_scrambling_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,534 copying lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,536 copying lm_eval/tasks/blimp_nl/adpositional_phrases__argument_scrambling.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,538 copying lm_eval/tasks/blimp_nl/anaphor_agreement__number.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,540 copying lm_eval/tasks/blimp_nl/wh_movement__stranding_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,543 copying lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,545 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,547 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,549 copying lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,552 copying lm_eval/tasks/blimp_nl/relativization__island.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,554 copying lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,556 copying lm_eval/tasks/blimp_nl/wh_movement__stranding_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,559 copying lm_eval/tasks/blimp_nl/extraposition__adjectival_adverbial.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,561 copying lm_eval/tasks/blimp_nl/complementive__transitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,563 copying lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,565 copying lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,567 copying lm_eval/tasks/blimp_nl/anaphor_agreement__person.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,570 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,572 copying lm_eval/tasks/blimp_nl/binding_principle_a__monomorphemic.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,574 copying lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,576 copying lm_eval/tasks/blimp_nl/passive__aci.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,578 copying lm_eval/tasks/blimp_nl/r_words__adverbial.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,581 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__resumptive_prolepsis.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,583 copying lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,585 copying lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,587 copying lm_eval/tasks/blimp_nl/auxiliaries__order_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,589 copying lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,591 copying lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,594 copying lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,596 copying lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,598 copying lm_eval/tasks/blimp_nl/complementive__ditransitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,600 copying lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,602 copying lm_eval/tasks/blimp_nl/finite_argument_clause__complementizer.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,605 copying lm_eval/tasks/blimp_nl/blimp_nl_group.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,607 copying lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,609 copying lm_eval/tasks/blimp_nl/extraposition__adjectival_supplementive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,612 copying lm_eval/tasks/blimp_nl/adverbial_modification__position_type.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,614 copying lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,616 copying lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,618 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__superiority.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,620 copying lm_eval/tasks/blimp_nl/extraposition__argument_nominal.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,623 copying lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,626 copying lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,630 copying lm_eval/tasks/blimp_nl/crossing_dependencies__cross_dependency.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,633 copying lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_2.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,637 copying lm_eval/tasks/blimp_nl/binding_principle_a__c_command.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,641 copying lm_eval/tasks/blimp_nl/determiners__geen_scrambling_1.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,644 copying lm_eval/tasks/blimp_nl/determiners__negative_polarity.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,647 copying lm_eval/tasks/blimp_nl/complementive__intransitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,650 copying lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,654 copying lm_eval/tasks/blimp_nl/finite_argument_clause__perception_dat.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,657 copying lm_eval/tasks/blimp_nl/passive__impersonal.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,660 copying lm_eval/tasks/blimp_nl/adverbial_modification__position_proform.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,662 copying lm_eval/tasks/blimp_nl/argument_structure__argument_number_ditransitive.yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:22,664 copying lm_eval/tasks/moral_stories/moral_stories.yaml -> build/lib/lm_eval/tasks/moral_stories 2026-05-11T15:52:22,666 copying lm_eval/tasks/lingoly/lingoly_context.yaml -> build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:22,668 copying lm_eval/tasks/lingoly/lingoly_group.yaml -> build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:22,670 copying lm_eval/tasks/lingoly/lingoly_nocontext.yaml -> build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:22,672 copying lm_eval/tasks/logiqa2/logieval.yaml -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T15:52:22,675 copying lm_eval/tasks/logiqa2/logiqa2.yaml -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T15:52:22,677 creating build/lib/lm_eval/tasks/fld 2026-05-11T15:52:22,678 copying lm_eval/tasks/fld/fld_star.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T15:52:22,680 copying lm_eval/tasks/fld/fld_logical_formula_star.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T15:52:22,682 copying lm_eval/tasks/fld/fld_logical_formula_default.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T15:52:22,684 copying lm_eval/tasks/fld/fld_default.yaml -> build/lib/lm_eval/tasks/fld 2026-05-11T15:52:22,686 copying lm_eval/tasks/darija_bench/darija_transliteration/translation_dr_ar.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:22,689 copying lm_eval/tasks/darija_bench/darija_transliteration/transliteration_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:22,691 copying lm_eval/tasks/darija_bench/darija_transliteration/transliteration_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:22,693 copying lm_eval/tasks/darija_bench/darija_transliteration/translation_ar_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:22,695 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msac.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:22,698 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_myc.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:22,700 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:22,702 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_electrom.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:22,705 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msda.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:22,707 copying lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_mac.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:22,709 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,711 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_fr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,714 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_en_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,716 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,718 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_msa.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,721 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,723 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,725 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,727 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_en.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,729 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_msa_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,732 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_msa.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,734 copying lm_eval/tasks/darija_bench/darija_translation/translation_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,736 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_dr_en.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,738 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,740 copying lm_eval/tasks/darija_bench/darija_translation/seed_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,743 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_fr_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,745 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_fr_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,747 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_fr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,749 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_dr_msa.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,752 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_en_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,754 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_msa_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,756 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_en.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,758 copying lm_eval/tasks/darija_bench/darija_translation/madar_translation_all.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,760 copying lm_eval/tasks/darija_bench/darija_translation/flores_translation_msa_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,762 copying lm_eval/tasks/darija_bench/darija_translation/doda_translation_en_dr.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:22,765 copying lm_eval/tasks/darija_bench/darija_summarization/summarization.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:52:22,767 copying lm_eval/tasks/darija_bench/darija_summarization/summarization_darija.yaml -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:52:22,769 creating build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,771 copying lm_eval/tasks/turkishmmlu/config_cot/History.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,773 copying lm_eval/tasks/turkishmmlu/config_cot/Turkish_Language_and_Literature.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,776 copying lm_eval/tasks/turkishmmlu/config_cot/Geography.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,779 copying lm_eval/tasks/turkishmmlu/config_cot/Mathematics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,781 copying lm_eval/tasks/turkishmmlu/config_cot/Biology.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,785 copying lm_eval/tasks/turkishmmlu/config_cot/Chemistry.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,788 copying lm_eval/tasks/turkishmmlu/config_cot/Philosophy.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,791 copying lm_eval/tasks/turkishmmlu/config_cot/Physics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,793 copying lm_eval/tasks/turkishmmlu/config_cot/Religion_and_Ethics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:22,796 creating build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,797 copying lm_eval/tasks/turkishmmlu/config/History.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,799 copying lm_eval/tasks/turkishmmlu/config/Turkish_Language_and_Literature.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,802 copying lm_eval/tasks/turkishmmlu/config/Geography.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,804 copying lm_eval/tasks/turkishmmlu/config/Mathematics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,806 copying lm_eval/tasks/turkishmmlu/config/Biology.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,808 copying lm_eval/tasks/turkishmmlu/config/Chemistry.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,810 copying lm_eval/tasks/turkishmmlu/config/Philosophy.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,813 copying lm_eval/tasks/turkishmmlu/config/Physics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,815 copying lm_eval/tasks/turkishmmlu/config/Religion_and_Ethics.yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:22,817 creating build/lib/lm_eval/tasks/polemo2 2026-05-11T15:52:22,818 copying lm_eval/tasks/polemo2/polemo2_in.yaml -> build/lib/lm_eval/tasks/polemo2 2026-05-11T15:52:22,821 copying lm_eval/tasks/polemo2/polemo2_out.yaml -> build/lib/lm_eval/tasks/polemo2 2026-05-11T15:52:22,823 copying lm_eval/tasks/mmlu_pro/mmlu_pro_history.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,826 copying lm_eval/tasks/mmlu_pro/mmlu_pro_business.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,830 copying lm_eval/tasks/mmlu_pro/mmlu_pro_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,832 copying lm_eval/tasks/mmlu_pro/_mmlu_pro.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,834 copying lm_eval/tasks/mmlu_pro/mmlu_pro_biology.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,836 copying lm_eval/tasks/mmlu_pro/mmlu_pro_health.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,839 copying lm_eval/tasks/mmlu_pro/mmlu_pro_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,841 copying lm_eval/tasks/mmlu_pro/mmlu_pro_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,843 copying lm_eval/tasks/mmlu_pro/mmlu_pro_engineering.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,845 copying lm_eval/tasks/mmlu_pro/mmlu_pro_economics.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,847 copying lm_eval/tasks/mmlu_pro/mmlu_pro_other.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,850 copying lm_eval/tasks/mmlu_pro/mmlu_pro_psychology.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,852 copying lm_eval/tasks/mmlu_pro/mmlu_pro_physics.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,854 copying lm_eval/tasks/mmlu_pro/mmlu_pro_law.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,856 copying lm_eval/tasks/mmlu_pro/mmlu_pro_math.yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:22,859 copying lm_eval/tasks/leaderboard/leaderboard.yaml -> build/lib/lm_eval/tasks/leaderboard 2026-05-11T15:52:22,861 creating build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,862 copying lm_eval/tasks/leaderboard/bbh_mc/causal_judgement.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,865 copying lm_eval/tasks/leaderboard/bbh_mc/salient_translation_error_detection.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,868 copying lm_eval/tasks/leaderboard/bbh_mc/navigate.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,870 copying lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_three_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,873 copying lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_seven_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,876 copying lm_eval/tasks/leaderboard/bbh_mc/temporal_sequences.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,878 copying lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_three_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,880 copying lm_eval/tasks/leaderboard/bbh_mc/_leaderboard_bbh.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,882 copying lm_eval/tasks/leaderboard/bbh_mc/geometric_shapes.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,885 copying lm_eval/tasks/leaderboard/bbh_mc/boolean_expressions.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,887 copying lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_five_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,889 copying lm_eval/tasks/leaderboard/bbh_mc/date_understanding.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,892 copying lm_eval/tasks/leaderboard/bbh_mc/web_of_lies.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,894 copying lm_eval/tasks/leaderboard/bbh_mc/snarks.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,896 copying lm_eval/tasks/leaderboard/bbh_mc/reasoning_about_colored_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,899 copying lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,901 copying lm_eval/tasks/leaderboard/bbh_mc/hyperbaton.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,903 copying lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_five_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,905 copying lm_eval/tasks/leaderboard/bbh_mc/sports_understanding.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,908 copying lm_eval/tasks/leaderboard/bbh_mc/object_counting.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,910 copying lm_eval/tasks/leaderboard/bbh_mc/disambiguation_qa.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,912 copying lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_seven_objects.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,915 copying lm_eval/tasks/leaderboard/bbh_mc/penguins_in_a_table.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,917 copying lm_eval/tasks/leaderboard/bbh_mc/movie_recommendation.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,919 copying lm_eval/tasks/leaderboard/bbh_mc/ruin_names.yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:22,922 copying lm_eval/tasks/leaderboard/math/math_prealgebra.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,924 copying lm_eval/tasks/leaderboard/math/math_geometry.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,926 copying lm_eval/tasks/leaderboard/math/math_algebra.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,928 copying lm_eval/tasks/leaderboard/math/math_intermediate_algebra.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,930 copying lm_eval/tasks/leaderboard/math/math_counting_and_prob.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,933 copying lm_eval/tasks/leaderboard/math/math_precalculus.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,935 copying lm_eval/tasks/leaderboard/math/_leaderboard_math.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,937 copying lm_eval/tasks/leaderboard/math/math_num_theory.yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:22,939 copying lm_eval/tasks/leaderboard/mmlu_pro/mmlu_pro.yaml -> build/lib/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T15:52:22,941 copying lm_eval/tasks/leaderboard/ifeval/_leaderboard_instruction_following.yaml -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:22,943 copying lm_eval/tasks/leaderboard/ifeval/ifeval.yaml -> build/lib/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:52:22,946 copying lm_eval/tasks/leaderboard/musr/musr_murder_mysteries.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:22,948 copying lm_eval/tasks/leaderboard/musr/_musr.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:22,950 copying lm_eval/tasks/leaderboard/musr/musr_team_allocation.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:22,952 copying lm_eval/tasks/leaderboard/musr/musr_object_placements.yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:22,955 copying lm_eval/tasks/leaderboard/gpqa/_leaderboard_gpqa.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:22,957 copying lm_eval/tasks/leaderboard/gpqa/gpqa_extended_zeroshot.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:22,959 copying lm_eval/tasks/leaderboard/gpqa/gpqa_main_zeroshot.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:22,961 copying lm_eval/tasks/leaderboard/gpqa/gpqa_diamond_zeroshot.yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:22,963 creating build/lib/lm_eval/tasks/sciq 2026-05-11T15:52:22,965 copying lm_eval/tasks/sciq/sciq.yaml -> build/lib/lm_eval/tasks/sciq 2026-05-11T15:52:22,967 copying lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,969 copying lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,972 copying lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,974 copying lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,976 copying lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,979 copying lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,981 copying lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,983 copying lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,985 copying lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,987 copying lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,990 copying lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,992 copying lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,994 copying lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,996 copying lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:22,998 copying lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,001 copying lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,003 copying lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,005 copying lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,007 copying lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,009 copying lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,012 copying lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,014 copying lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,016 copying lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,019 copying lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,021 copying lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,023 copying lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,025 copying lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,028 copying lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,030 copying lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,032 copying lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,035 copying lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,037 copying lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,039 copying lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,041 copying lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,044 copying lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,046 copying lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,049 copying lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,051 copying lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,053 copying lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,055 copying lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,058 copying lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,060 copying lm_eval/tasks/ceval/_ceval-valid.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,063 copying lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,065 copying lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,067 copying lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,069 copying lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,071 copying lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,073 copying lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,075 copying lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,078 copying lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,080 copying lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,082 copying lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,084 copying lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:23,087 copying lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,089 copying lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,091 copying lm_eval/tasks/aclue/_aclue.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,093 copying lm_eval/tasks/aclue/aclue_ancient_literature.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,095 copying lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,098 copying lm_eval/tasks/aclue/aclue_ancient_medical.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,100 copying lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,102 copying lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,104 copying lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,106 copying lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,109 copying lm_eval/tasks/aclue/aclue_couplet_prediction.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,111 copying lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,113 copying lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,116 copying lm_eval/tasks/aclue/aclue_reading_comprehension.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,118 copying lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,120 copying lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:23,122 copying lm_eval/tasks/webqs/webqs.yaml -> build/lib/lm_eval/tasks/webqs 2026-05-11T15:52:23,125 copying lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:23,127 copying lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:23,129 copying lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:23,132 copying lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:23,134 copying lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:23,136 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T15:52:23,137 copying lm_eval/tasks/afrimgsm/direct_cot/afrimgsm_cot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T15:52:23,139 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,140 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,143 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,145 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,147 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,149 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,152 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,154 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,156 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,158 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,161 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,163 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,165 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,167 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,170 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,172 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,174 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,176 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,179 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,181 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:23,183 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,184 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,187 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,189 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,192 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,194 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,196 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,199 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,201 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,203 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,205 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,207 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,210 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,212 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,214 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,216 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,218 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,221 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,223 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,225 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:23,227 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,228 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,231 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,233 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,235 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,238 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,240 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,242 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,244 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,247 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,249 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,251 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,253 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,256 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,258 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,260 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,262 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,264 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,267 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,269 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:23,271 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,272 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,275 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,277 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,279 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,282 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,284 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,286 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,289 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,291 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,293 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,296 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,298 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,300 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,303 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,305 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,307 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,310 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,312 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,314 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:23,316 creating build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,318 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,320 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,322 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,325 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,327 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,329 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,332 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,334 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,336 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,339 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,341 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,343 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,346 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,348 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,350 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,353 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,355 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,358 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,360 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:23,362 creating build/lib/lm_eval/tasks/afrimgsm/translate 2026-05-11T15:52:23,364 copying lm_eval/tasks/afrimgsm/translate/afrimgsm_tt.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate 2026-05-11T15:52:23,366 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,367 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,370 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,372 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,374 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,377 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,379 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,381 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,383 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,386 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,388 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,390 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,393 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,395 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,397 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,399 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,402 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,404 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:23,406 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,407 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,410 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,412 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,414 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,417 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,420 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,422 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,424 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,427 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,429 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,431 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,433 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,436 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,438 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,440 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,442 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,444 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:23,446 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,447 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,450 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,452 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,455 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,457 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,459 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,461 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,463 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,466 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,468 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,470 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,472 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,475 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,477 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,479 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,482 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,484 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:23,486 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,487 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,490 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,492 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,494 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,496 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,499 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,501 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,503 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,505 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,507 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,509 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,512 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,514 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,516 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,518 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,521 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,523 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:23,525 creating build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,526 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,529 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,531 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,533 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,535 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,537 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,540 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,542 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,545 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,547 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,549 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,552 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,554 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,556 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,558 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,560 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,562 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:23,565 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T15:52:23,566 copying lm_eval/tasks/afrimgsm/translate_cot/afrimgsm_tt_cot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T15:52:23,568 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,569 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,571 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,573 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,576 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,578 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,580 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,582 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,584 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,587 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,589 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,591 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,594 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,596 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,598 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,601 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,603 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,605 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,608 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:23,610 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,611 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,613 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,615 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,617 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,619 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,622 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,624 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,626 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,628 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,630 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,633 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,635 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,637 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,639 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,642 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,644 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,646 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,649 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:23,651 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,652 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,654 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,657 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,659 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,661 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,663 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,666 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,668 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,670 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,673 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,675 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,677 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,679 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,681 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,684 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,686 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,689 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,691 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:23,693 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,694 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,697 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,699 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,701 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,703 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,706 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,708 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,710 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,712 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,715 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,717 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,719 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,721 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,723 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,725 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,728 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,730 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,732 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:23,734 creating build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,735 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,737 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,740 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,742 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,744 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,746 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,749 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,751 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,753 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,756 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,758 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,760 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,762 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,765 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,767 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,769 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,772 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,774 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:23,776 creating build/lib/lm_eval/tasks/afrimgsm/direct 2026-05-11T15:52:23,777 copying lm_eval/tasks/afrimgsm/direct/afrimgsm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct 2026-05-11T15:52:23,779 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,780 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,782 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,784 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,787 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,789 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,791 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,793 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,796 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,798 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,800 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,802 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,804 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,807 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,809 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,811 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,814 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,816 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,818 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,820 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:23,822 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,824 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,826 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,828 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,831 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,833 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,835 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,837 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,839 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,841 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,844 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,846 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,848 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,850 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,852 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,854 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,857 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,859 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,861 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,863 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:23,865 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,867 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,869 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,871 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,873 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,876 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,878 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,880 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,882 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,885 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,887 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,889 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,891 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,894 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,896 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,898 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,900 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,902 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,904 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,907 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:23,909 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,910 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,912 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,915 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,917 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,919 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,921 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,924 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,926 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,928 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,930 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,933 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,935 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,937 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,939 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,941 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,943 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,945 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,948 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,950 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:23,952 creating build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,954 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sot.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,956 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_orm.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,958 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_twi.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,960 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,962 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sna.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,964 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_vai.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,966 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ewe.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,969 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yor.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,971 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_fra.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,973 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_kin.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,975 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_amh.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,977 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_hau.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,980 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_wol.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,982 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ibo.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,985 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_swa.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,987 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lug.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,990 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_eng.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,992 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_xho.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,994 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_zul.yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:23,996 creating build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:23,997 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,000 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,002 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,004 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,006 copying lm_eval/tasks/bertaqa/bertaqa_en.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,009 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,011 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,013 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,015 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,017 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,020 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,023 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,025 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,027 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,030 copying lm_eval/tasks/bertaqa/bertaqa_eu.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,033 copying lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:24,035 creating build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:24,036 copying lm_eval/tasks/unscramble/cycle_letters.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:24,038 copying lm_eval/tasks/unscramble/anagrams2.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:24,040 copying lm_eval/tasks/unscramble/reversed_words.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:24,042 copying lm_eval/tasks/unscramble/random_insertion.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:24,045 copying lm_eval/tasks/unscramble/anagrams1.yaml -> build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:24,047 creating build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,048 copying lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,050 copying lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,053 copying lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,055 copying lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,057 copying lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,059 copying lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,061 copying lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,064 copying lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,067 copying lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,070 copying lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,073 copying lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,076 copying lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:24,078 creating build/lib/lm_eval/tasks/openbookqa 2026-05-11T15:52:24,079 copying lm_eval/tasks/openbookqa/openbookqa.yaml -> build/lib/lm_eval/tasks/openbookqa 2026-05-11T15:52:24,082 creating build/lib/lm_eval/tasks/nq_open 2026-05-11T15:52:24,083 copying lm_eval/tasks/nq_open/nq_open.yaml -> build/lib/lm_eval/tasks/nq_open 2026-05-11T15:52:24,085 copying lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,087 copying lm_eval/tasks/xwinograd/_xwinograd.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,090 copying lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,092 copying lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,094 copying lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,097 copying lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,099 copying lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:24,102 copying lm_eval/tasks/med_prescriptions/med_prescriptions_hard.yaml -> build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T15:52:24,104 copying lm_eval/tasks/med_prescriptions/med_prescriptions_easy.yaml -> build/lib/lm_eval/tasks/med_prescriptions 2026-05-11T15:52:24,107 copying lm_eval/tasks/afrixnli/translate/afrixnli_tt.yaml -> build/lib/lm_eval/tasks/afrixnli/translate 2026-05-11T15:52:24,109 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,111 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,114 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,116 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,118 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,120 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,122 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,125 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,127 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,130 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,132 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,134 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,136 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,139 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,141 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,143 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,145 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:24,147 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,150 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,152 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,154 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,156 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,159 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,161 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,163 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,166 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,168 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,170 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,172 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,174 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,176 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,179 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,181 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,183 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:24,185 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,187 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,189 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,192 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,194 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,196 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,199 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,201 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,203 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,205 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,207 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,210 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,212 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,214 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,217 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,219 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,221 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:24,224 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,226 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,228 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,230 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,233 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,235 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,237 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,239 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,242 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,244 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,246 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,248 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,251 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,253 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,255 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,257 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,260 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:24,262 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,265 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,267 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,269 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,272 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,274 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,276 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,278 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,280 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,283 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,285 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,287 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,289 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,291 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,294 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,296 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,298 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:24,300 copying lm_eval/tasks/afrixnli/direct/afrixnli.yaml -> build/lib/lm_eval/tasks/afrixnli/direct 2026-05-11T15:52:24,303 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,305 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,308 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,310 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,312 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,315 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,317 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,319 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,322 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,324 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,326 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,328 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,330 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,333 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,335 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,337 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,339 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,342 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:24,344 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,346 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,348 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,351 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,353 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,355 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,357 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,359 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,361 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,363 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,366 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,368 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,370 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,373 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,375 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,377 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,379 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,382 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:24,384 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,386 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,388 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,390 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,392 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,394 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,396 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,399 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,401 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,403 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,405 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,407 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,409 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,411 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,413 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,415 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,417 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,419 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:24,421 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,423 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,424 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,426 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,428 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,430 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,432 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,434 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,437 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,439 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,441 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,443 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,445 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,447 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,449 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,451 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,453 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,455 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:24,457 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,459 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,461 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,464 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,466 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,468 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,470 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,472 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,474 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,476 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,479 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,481 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,483 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,485 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,487 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,489 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,492 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,494 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:24,496 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,499 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,501 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,503 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,505 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,507 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,509 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,512 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,514 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,516 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,518 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,520 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,522 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,524 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,527 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,529 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,531 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:24,533 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,536 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,538 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,540 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,542 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,544 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,546 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,548 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,551 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,553 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,555 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,557 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,559 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,561 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,564 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,566 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,568 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,570 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:24,573 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,575 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,577 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,580 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,582 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,584 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,587 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,589 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,591 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,593 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,596 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,600 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,602 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,604 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,607 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,609 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,611 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,614 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:24,616 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,618 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,620 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,622 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,625 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,627 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,629 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,631 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,634 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,636 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,639 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,641 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,643 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,645 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,648 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,651 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,653 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:24,655 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,657 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,659 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,662 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,664 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,666 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,668 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,671 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,673 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,675 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,678 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,680 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,682 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,685 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,687 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,689 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,691 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,694 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:24,696 creating build/lib/lm_eval/tasks/siqa 2026-05-11T15:52:24,697 copying lm_eval/tasks/siqa/siqa.yaml -> build/lib/lm_eval/tasks/siqa 2026-05-11T15:52:24,700 copying lm_eval/tasks/agieval/logiqa-zh.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,702 copying lm_eval/tasks/agieval/agieval_nous.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,704 copying lm_eval/tasks/agieval/gaokao-chinese.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,706 copying lm_eval/tasks/agieval/agieval_cn.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,709 copying lm_eval/tasks/agieval/gaokao-chemistry.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,711 copying lm_eval/tasks/agieval/aqua-rat.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,713 copying lm_eval/tasks/agieval/jec-qa-kd.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,715 copying lm_eval/tasks/agieval/gaokao-history.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,718 copying lm_eval/tasks/agieval/gaokao-english.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,720 copying lm_eval/tasks/agieval/gaokao-physics.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,722 copying lm_eval/tasks/agieval/math.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,724 copying lm_eval/tasks/agieval/lsat-lr.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,727 copying lm_eval/tasks/agieval/sat-en.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,729 copying lm_eval/tasks/agieval/gaokao-mathqa.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,731 copying lm_eval/tasks/agieval/sat-math.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,733 copying lm_eval/tasks/agieval/sat-en-without-passage.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,736 copying lm_eval/tasks/agieval/jec-qa-ca.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,738 copying lm_eval/tasks/agieval/gaokao-mathcloze.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,740 copying lm_eval/tasks/agieval/agieval_en.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,743 copying lm_eval/tasks/agieval/agieval.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,745 copying lm_eval/tasks/agieval/gaokao-biology.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,747 copying lm_eval/tasks/agieval/logiqa-en.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,750 copying lm_eval/tasks/agieval/gaokao-geography.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,752 copying lm_eval/tasks/agieval/lsat-ar.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,754 copying lm_eval/tasks/agieval/lsat-rc.yaml -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:24,757 creating build/lib/lm_eval/tasks/babi 2026-05-11T15:52:24,758 copying lm_eval/tasks/babi/babi.yaml -> build/lib/lm_eval/tasks/babi 2026-05-11T15:52:24,760 copying lm_eval/tasks/ifeval/ifeval.yaml -> build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:24,763 copying lm_eval/tasks/ifeval/multilingual/ifeval_ca.yaml -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T15:52:24,765 copying lm_eval/tasks/ifeval/multilingual/ifeval_es.yaml -> build/lib/lm_eval/tasks/ifeval/multilingual 2026-05-11T15:52:24,767 creating build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,769 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,771 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,773 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,776 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,778 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,780 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,782 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,784 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,787 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,789 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,791 copying lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:24,793 creating build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,794 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,797 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,799 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,801 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,803 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,806 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,809 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,811 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,813 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,815 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,818 copying lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:24,820 creating build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,821 copying lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,823 copying lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,827 copying lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,829 copying lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,832 copying lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,834 copying lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,836 copying lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,838 copying lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,841 copying lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,843 copying lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,845 copying lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:24,847 creating build/lib/lm_eval/tasks/asdiv 2026-05-11T15:52:24,849 copying lm_eval/tasks/asdiv/asdiv-cot-llama.yaml -> build/lib/lm_eval/tasks/asdiv 2026-05-11T15:52:24,852 copying lm_eval/tasks/asdiv/default.yaml -> build/lib/lm_eval/tasks/asdiv 2026-05-11T15:52:24,854 creating build/lib/lm_eval/tasks/lambada 2026-05-11T15:52:24,855 copying lm_eval/tasks/lambada/lambada_openai.yaml -> build/lib/lm_eval/tasks/lambada 2026-05-11T15:52:24,857 copying lm_eval/tasks/lambada/lambada_standard.yaml -> build/lib/lm_eval/tasks/lambada 2026-05-11T15:52:24,859 creating build/lib/lm_eval/tasks/triviaqa 2026-05-11T15:52:24,861 copying lm_eval/tasks/triviaqa/default.yaml -> build/lib/lm_eval/tasks/triviaqa 2026-05-11T15:52:24,863 copying lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,865 copying lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,867 copying lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,869 copying lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,872 copying lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,874 copying lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,876 copying lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,878 copying lm_eval/tasks/tinyBenchmarks/tinyArc.yaml -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:24,880 copying lm_eval/tasks/humaneval_infilling/single_line_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:24,883 copying lm_eval/tasks/humaneval_infilling/random_span_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:24,885 copying lm_eval/tasks/humaneval_infilling/multi_line_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:24,887 copying lm_eval/tasks/humaneval_infilling/random_span_infilling_light.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:24,890 copying lm_eval/tasks/humaneval_infilling/humaneval_infilling.yaml -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:24,892 copying lm_eval/tasks/arabicmmlu/_arabicmmlu.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,894 copying lm_eval/tasks/arabicmmlu/arabicmmlu_history_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,896 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,899 copying lm_eval/tasks/arabicmmlu/arabicmmlu_philosophy_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,901 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,903 copying lm_eval/tasks/arabicmmlu/arabicmmlu_geography_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,906 copying lm_eval/tasks/arabicmmlu/arabicmmlu_economics_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,909 copying lm_eval/tasks/arabicmmlu/arabicmmlu_geography_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,911 copying lm_eval/tasks/arabicmmlu/arabicmmlu_political_science_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,913 copying lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,916 copying lm_eval/tasks/arabicmmlu/arabicmmlu_economics_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,919 copying lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,921 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_humanities.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,924 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,926 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,929 copying lm_eval/tasks/arabicmmlu/arabicmmlu_law_professional.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,932 copying lm_eval/tasks/arabicmmlu/arabicmmlu_management_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,934 copying lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,937 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,939 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,942 copying lm_eval/tasks/arabicmmlu/arabicmmlu_history_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,944 copying lm_eval/tasks/arabicmmlu/arabicmmlu_history_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,947 copying lm_eval/tasks/arabicmmlu/arabicmmlu_civics_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,949 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,952 copying lm_eval/tasks/arabicmmlu/arabicmmlu_math_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,954 copying lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,957 copying lm_eval/tasks/arabicmmlu/arabicmmlu_geography_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,959 copying lm_eval/tasks/arabicmmlu/arabicmmlu_physics_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,962 copying lm_eval/tasks/arabicmmlu/arabicmmlu_economics_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,964 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,967 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,970 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,972 copying lm_eval/tasks/arabicmmlu/arabicmmlu_accounting_university.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,975 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_language.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,977 copying lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,980 copying lm_eval/tasks/arabicmmlu/arabicmmlu_civics_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,983 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_other.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,985 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,988 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_primary_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,990 copying lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,993 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_social_science.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,995 copying lm_eval/tasks/arabicmmlu/arabicmmlu_biology_high_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:24,998 copying lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:25,001 copying lm_eval/tasks/arabicmmlu/_arabicmmlu_stem.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:25,003 copying lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_middle_school.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:25,006 copying lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:25,008 copying lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,011 copying lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,014 copying lm_eval/tasks/french_bench/french_bench_boolqa.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,016 copying lm_eval/tasks/french_bench/french_bench_xnli.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,019 copying lm_eval/tasks/french_bench/french_bench_multifquad.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,021 copying lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,024 copying lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,026 copying lm_eval/tasks/french_bench/french_bench_grammar.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,029 copying lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,032 copying lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,034 copying lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,037 copying lm_eval/tasks/french_bench/french_bench_reading_comp.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,039 copying lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,042 copying lm_eval/tasks/french_bench/french_bench_trivia.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,044 copying lm_eval/tasks/french_bench/french_bench_fquadv2.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,047 copying lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,050 copying lm_eval/tasks/french_bench/french_bench_vocab.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,052 copying lm_eval/tasks/french_bench/french_bench_hellaswag.yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:25,055 creating build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,056 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,059 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,061 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,064 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,066 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,069 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__sing_MS_MV_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,072 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,074 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,077 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,080 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,082 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,085 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,087 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,090 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,092 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,094 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,097 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_plur_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,099 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,102 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,104 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,107 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,110 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,112 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__plur_MS_ANPHR.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,115 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,117 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,120 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,122 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,125 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__plur_MS_MV_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,127 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,130 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,132 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,135 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,137 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,140 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,143 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,145 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,148 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,151 copying lm_eval/tasks/lm_syneval/lm_syneval_group.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,154 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,157 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,159 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,162 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,164 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,167 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,170 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,172 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,175 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,177 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,180 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,183 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,186 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__plur_MS_LMV_LMV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,189 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__future.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,192 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,194 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__sing_MS_LMV_LMV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,197 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_sing_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,199 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,202 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_sing_BS.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,204 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,207 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,210 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,212 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,215 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,217 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,220 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,223 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__sing_MS_ANPHR.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,226 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,229 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,232 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_plur_ES.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,234 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,237 copying lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_plur_ES_EV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,239 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,242 copying lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,245 copying lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__past.yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:25,247 copying lm_eval/tasks/egymmlu/_egymmlu_mmlu.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,250 copying lm_eval/tasks/egymmlu/egymmlu_management.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,253 copying lm_eval/tasks/egymmlu/egymmlu_driving_test.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,255 copying lm_eval/tasks/egymmlu/egymmlu_security_studies.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,258 copying lm_eval/tasks/egymmlu/egymmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,261 copying lm_eval/tasks/egymmlu/egymmlu_professional_law.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,263 copying lm_eval/tasks/egymmlu/egymmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,266 copying lm_eval/tasks/egymmlu/egymmlu_islamic_studies.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,270 copying lm_eval/tasks/egymmlu/egymmlu_arabic_language_(general).yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,273 copying lm_eval/tasks/egymmlu/egymmlu_economics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,276 copying lm_eval/tasks/egymmlu/egymmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,279 copying lm_eval/tasks/egymmlu/egymmlu_accounting.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,281 copying lm_eval/tasks/egymmlu/egymmlu_nutrition.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,283 copying lm_eval/tasks/egymmlu/egymmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,286 copying lm_eval/tasks/egymmlu/egymmlu_world_religions.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,288 copying lm_eval/tasks/egymmlu/egymmlu_biology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,290 copying lm_eval/tasks/egymmlu/egymmlu_physics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,292 copying lm_eval/tasks/egymmlu/egymmlu_philosophy_ar.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,294 copying lm_eval/tasks/egymmlu/egymmlu_sociology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,297 copying lm_eval/tasks/egymmlu/egymmlu_arabic_language_(grammar).yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,299 copying lm_eval/tasks/egymmlu/egymmlu_philosophy.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,301 copying lm_eval/tasks/egymmlu/egymmlu_math.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,303 copying lm_eval/tasks/egymmlu/egymmlu_international_law.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,305 copying lm_eval/tasks/egymmlu/egymmlu_political_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,308 copying lm_eval/tasks/egymmlu/egymmlu_computer_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,310 copying lm_eval/tasks/egymmlu/egymmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,312 copying lm_eval/tasks/egymmlu/egymmlu_global_facts.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,314 copying lm_eval/tasks/egymmlu/egymmlu_natural_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,316 copying lm_eval/tasks/egymmlu/egymmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,318 copying lm_eval/tasks/egymmlu/egymmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,321 copying lm_eval/tasks/egymmlu/_egymmlu_ar_mmlu.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,323 copying lm_eval/tasks/egymmlu/egymmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,325 copying lm_eval/tasks/egymmlu/egymmlu_arabic_language.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,327 copying lm_eval/tasks/egymmlu/egymmlu_civics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,329 copying lm_eval/tasks/egymmlu/egymmlu_law.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,331 copying lm_eval/tasks/egymmlu/egymmlu_management_ar.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,334 copying lm_eval/tasks/egymmlu/egymmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,336 copying lm_eval/tasks/egymmlu/egymmlu_general_knowledge.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,338 copying lm_eval/tasks/egymmlu/egymmlu_social_science.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,340 copying lm_eval/tasks/egymmlu/egymmlu_history.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,343 copying lm_eval/tasks/egymmlu/egymmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,345 copying lm_eval/tasks/egymmlu/egymmlu_geography.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,347 copying lm_eval/tasks/egymmlu/egymmlu_public_relations.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,350 copying lm_eval/tasks/egymmlu/egymmlu_marketing.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,352 copying lm_eval/tasks/egymmlu/egymmlu_human_aging.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,354 copying lm_eval/tasks/egymmlu/_egymmlu.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,356 copying lm_eval/tasks/egymmlu/egymmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:25,359 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,361 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,363 copying lm_eval/tasks/global_mmlu/default/de/_global_mmlu_de.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,365 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,368 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,370 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,372 copying lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:25,374 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,377 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,379 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,381 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,383 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,386 copying lm_eval/tasks/global_mmlu/default/yo/_global_mmlu_yo.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,388 copying lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:25,390 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,392 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,394 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,397 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,399 copying lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,401 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,403 copying lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:25,405 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,408 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,410 copying lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,412 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,414 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,416 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,419 copying lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:25,421 copying lm_eval/tasks/global_mmlu/default/bn/_global_mmlu_bn.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,423 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,425 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,428 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,430 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,432 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,434 copying lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:25,437 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,439 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,441 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,443 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,445 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,447 copying lm_eval/tasks/global_mmlu/default/en/_global_mmlu_en.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,450 copying lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:25,452 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,454 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,456 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,459 copying lm_eval/tasks/global_mmlu/default/sw/_global_mmlu_sw.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,461 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,463 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,466 copying lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:25,468 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,470 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,472 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,475 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,477 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,479 copying lm_eval/tasks/global_mmlu/default/es/_global_mmlu_es.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,482 copying lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:25,484 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,486 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,488 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,491 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,493 copying lm_eval/tasks/global_mmlu/default/ja/_global_mmlu_ja.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,495 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,497 copying lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:25,499 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,501 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,504 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,506 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,508 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,510 copying lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,512 copying lm_eval/tasks/global_mmlu/default/hi/_global_mmlu_hi.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:25,515 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,517 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,519 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,522 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,524 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,526 copying lm_eval/tasks/global_mmlu/default/it/_global_mmlu_it.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,528 copying lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:25,530 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,533 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,535 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,537 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,540 copying lm_eval/tasks/global_mmlu/default/fr/_global_mmlu_fr.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,542 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,544 copying lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:25,546 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,549 copying lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,551 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,553 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,556 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,558 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,560 copying lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:25,562 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,564 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,567 copying lm_eval/tasks/global_mmlu/default/id/_global_mmlu_id.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,569 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,571 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,573 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,575 copying lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:25,578 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,580 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,582 copying lm_eval/tasks/global_mmlu/default/ar/_global_mmlu_ar.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,585 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_business.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,587 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,589 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,592 copying lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_medical.yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:25,594 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,596 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,599 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,601 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,603 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,605 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,607 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,610 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,612 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,614 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,616 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,619 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,621 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,623 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,625 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,627 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,630 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,632 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,634 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,636 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,639 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,641 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,643 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,645 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,648 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,650 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,652 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,654 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,657 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,659 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,662 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,664 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,666 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,668 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,671 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,673 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,675 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,677 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,679 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,682 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,684 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,686 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,688 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,690 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,693 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,695 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,697 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,699 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,702 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,704 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,706 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,708 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,711 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,713 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,715 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,717 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,720 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,722 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,724 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,727 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,729 copying lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,732 copying lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:25,734 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,736 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,739 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,741 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,743 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,745 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,748 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,750 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,752 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,754 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,757 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,759 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,761 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,763 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,765 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,767 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,769 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,771 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,774 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,776 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,778 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,781 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,783 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,785 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,788 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,790 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,792 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,794 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,797 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,799 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,801 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,803 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,806 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,808 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,810 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,812 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,815 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,817 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,819 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,822 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,824 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,826 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,828 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,831 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,833 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,835 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,838 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,840 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,842 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,844 copying lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,847 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,849 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,851 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,853 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,855 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,858 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,860 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,862 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,864 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,866 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,869 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,871 copying lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:25,873 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,875 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,877 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,880 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,882 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,884 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,887 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,889 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,891 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,894 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,896 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,898 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,900 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,903 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,905 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,908 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,910 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,912 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,914 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,917 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,919 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,921 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,924 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,926 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,928 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,931 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,933 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,935 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,937 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,940 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,942 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,944 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,946 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,949 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,951 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,953 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,955 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,958 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,960 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,962 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,965 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,967 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,969 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,971 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,973 copying lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,976 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,978 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,980 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,982 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,984 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,987 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,989 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,991 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,993 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,996 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:25,998 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,000 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,003 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,005 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,007 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,009 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,012 copying lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:26,014 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,016 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,018 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,021 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,023 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,025 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,027 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,030 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,032 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,034 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,036 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,038 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,041 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,043 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,045 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,047 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,050 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,052 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,054 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,056 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,058 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,061 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,063 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,065 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,068 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,070 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,072 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,074 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,077 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,079 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,081 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,083 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,086 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,088 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,090 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,092 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,094 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,097 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,099 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,101 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,103 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,105 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,107 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,110 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,112 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,114 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,117 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,119 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,121 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,124 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,126 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,128 copying lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,130 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,132 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,135 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,137 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,139 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,142 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,144 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,146 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,148 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,151 copying lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:26,153 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,155 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,157 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,159 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,161 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,164 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,166 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,168 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,171 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,173 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,176 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,178 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,180 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,182 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,185 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,187 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,189 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,192 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,194 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,196 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,198 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,201 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,203 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,205 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,207 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,210 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,212 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,214 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,216 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,219 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,221 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,223 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,226 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,228 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,230 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,233 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,235 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,237 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,240 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,242 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,244 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,246 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,249 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,251 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,253 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,255 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,258 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,260 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,262 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,264 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,266 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,269 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,271 copying lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,273 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,275 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,277 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,280 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,282 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,284 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,286 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,289 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,291 copying lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:26,293 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,296 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,298 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,300 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,302 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,305 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,307 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,309 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,312 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,314 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,316 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,318 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,321 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,323 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,325 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,327 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,329 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,332 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,334 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,336 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,338 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,340 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,342 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,345 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,347 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,349 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,351 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,354 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,356 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,358 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,360 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,363 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,365 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,367 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,370 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,372 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,375 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,377 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,379 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,381 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,384 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,386 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,389 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,391 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,394 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,396 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,398 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,400 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,403 copying lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,405 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,407 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,410 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,412 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,414 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,416 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,419 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,421 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,424 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,427 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,430 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,432 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,434 copying lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:26,437 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,439 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,441 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,444 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,446 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,449 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,451 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,453 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,455 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,457 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,460 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,463 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,466 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,469 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,471 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,475 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,477 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,479 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,481 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,484 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,486 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,488 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,491 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,493 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,496 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,498 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,500 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,502 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,504 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,506 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,509 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,511 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,513 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,515 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,517 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,520 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,522 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,524 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,526 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,529 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,531 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,533 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,535 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,538 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,540 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,542 copying lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,544 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,547 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,549 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,551 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,554 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,556 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,558 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,560 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,563 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,565 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,567 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,569 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,571 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,574 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,576 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,578 copying lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:26,580 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,582 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,585 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,587 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,589 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,592 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,594 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,596 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,598 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,601 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,603 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,605 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,607 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,610 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,612 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,614 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,617 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,619 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,621 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,623 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,625 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,628 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,630 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,632 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,635 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,637 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,639 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,641 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,643 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,646 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,648 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,650 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,652 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,655 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,657 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,659 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,661 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,663 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,666 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,668 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,670 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,673 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,675 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,677 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,679 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,681 copying lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,684 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,686 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,688 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,690 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,692 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,695 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,697 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,699 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,701 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,703 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,706 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,708 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,710 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,712 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,714 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,716 copying lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:26,719 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,721 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,723 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,726 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,728 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,730 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,733 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,735 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,737 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,739 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,741 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,743 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,746 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,748 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,750 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,752 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,755 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,757 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,759 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,761 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,764 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,766 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,768 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,770 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,772 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,775 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,777 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,779 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,781 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,784 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,786 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,788 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,791 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,793 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,795 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,797 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,799 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,801 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,803 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,805 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,808 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,810 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,812 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,814 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,816 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,819 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,821 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,823 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,825 copying lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,828 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,830 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,832 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,835 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,837 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,839 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,841 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,844 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,846 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,849 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,851 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,853 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,855 copying lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:26,857 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,860 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,862 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,864 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,866 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,869 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,871 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,873 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,875 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,878 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,880 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,882 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,884 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,887 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,889 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,891 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,894 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,896 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,898 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,901 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,903 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,905 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,908 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,910 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,912 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,914 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,916 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,919 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,921 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,923 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,926 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,928 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,930 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,932 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,934 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,936 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,939 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,941 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,943 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,945 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,948 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,950 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,952 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,955 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,957 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,959 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,961 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,964 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,966 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,968 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,970 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,973 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,975 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,977 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,979 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,981 copying lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,984 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,986 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,989 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,991 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,993 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,995 copying lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:26,997 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,000 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,002 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,004 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,007 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,009 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,011 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,014 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,016 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,019 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,021 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,023 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,026 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,028 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,030 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,032 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,034 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,037 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,039 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,042 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,044 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,046 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,048 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,050 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,053 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,055 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,057 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,059 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,062 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,064 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,066 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,068 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,071 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,073 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,075 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,077 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,079 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,082 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,084 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,086 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,088 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,091 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,093 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,095 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,097 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,100 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,102 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,104 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,106 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,109 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,111 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,113 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,115 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,117 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,119 copying lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,122 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,124 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,126 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,128 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,130 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,133 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,135 copying lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:27,137 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,140 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,142 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,144 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,146 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,148 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,150 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,152 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,154 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,156 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,159 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,161 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,163 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,165 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,167 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,170 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,172 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,175 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,177 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,179 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,183 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,186 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,189 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,192 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,194 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,197 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,200 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,203 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,205 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,208 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,212 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,215 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,218 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,221 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,224 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,227 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,229 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,231 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,233 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,235 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,237 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,240 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,242 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,244 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,246 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,249 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,251 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,253 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,255 copying lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,257 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,259 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,261 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,263 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,265 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,268 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,270 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,272 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,274 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,276 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,278 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,280 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,282 copying lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:27,284 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,286 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,288 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,291 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,294 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,296 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,299 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,304 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,308 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,312 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,315 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,319 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,322 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,326 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,329 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,331 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,333 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,335 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,338 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,341 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,343 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,345 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,347 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,349 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,351 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,353 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,355 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,357 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,360 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,362 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,364 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,366 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,368 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,370 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,372 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,374 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,376 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,378 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,380 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,382 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,384 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,387 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,389 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,391 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,393 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,395 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,398 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,400 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,402 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,404 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,406 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,408 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,410 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,412 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,414 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,416 copying lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,418 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,420 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,422 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,424 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,426 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,428 copying lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:27,431 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,433 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,435 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,437 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,439 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,441 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,443 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,445 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,447 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,449 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,451 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,453 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,455 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,457 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,459 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,461 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,463 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,465 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,467 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,469 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,471 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,474 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,476 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,478 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,480 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,482 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,484 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,486 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,488 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,490 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,492 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,494 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,496 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,499 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,501 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,503 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,505 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,507 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,509 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,511 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,513 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,515 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,517 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,519 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,521 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,523 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,525 copying lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,527 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,529 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,531 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,534 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,536 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,538 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,540 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,542 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,544 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,546 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,548 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,551 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,553 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,555 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,557 copying lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:27,559 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,561 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,563 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,565 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,567 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,569 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,571 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,573 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,575 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,577 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,579 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,582 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,584 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,587 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,589 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,591 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,593 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,596 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,598 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,600 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,602 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,604 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,606 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,608 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,610 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,612 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,614 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,616 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,618 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,621 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,623 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,625 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,629 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,631 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,633 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,635 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,637 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,639 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,641 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,644 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,646 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,648 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,650 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,652 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,654 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,656 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,658 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,660 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,663 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,665 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,667 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,670 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,672 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,675 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,677 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,679 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,682 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,684 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,686 copying lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,688 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,690 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,692 copying lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:27,694 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,696 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,698 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,700 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,702 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,704 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,706 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,708 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,710 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,712 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,715 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,717 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,719 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,721 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,723 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,725 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,727 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,729 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,731 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,733 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,735 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,737 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,739 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,741 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,744 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,746 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,748 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,750 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,752 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,755 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,757 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,759 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,761 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,763 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,765 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,767 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,769 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,771 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,773 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,776 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,778 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,780 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,782 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,784 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,786 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,788 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,790 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,792 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,794 copying lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,796 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,798 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,800 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,802 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,804 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,806 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,808 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,810 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,813 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,815 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,817 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,819 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,821 copying lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:27,823 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,825 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,827 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,829 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,831 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,833 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,835 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,837 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,839 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,841 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,843 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,845 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,847 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,850 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,852 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,854 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,856 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,858 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,860 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,862 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,864 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,866 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,869 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,871 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,873 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,875 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,877 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,879 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,881 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,883 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,885 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,887 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,889 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,891 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,893 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,895 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,897 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,899 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,901 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,903 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,905 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,907 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,909 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,911 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,913 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,915 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,917 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,919 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,921 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,923 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,926 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,928 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,930 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,932 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,934 copying lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,936 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,938 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,940 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,942 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,944 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,947 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,949 copying lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:27,951 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,953 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,955 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,957 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,959 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,961 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,963 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,965 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,967 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,969 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,971 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,973 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,975 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,977 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,980 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,982 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,984 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,986 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,988 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,991 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,993 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,995 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,997 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:27,999 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,001 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,003 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,005 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,007 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,010 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,012 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,014 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,016 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,018 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,020 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,022 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,025 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,027 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,029 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,031 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,033 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,035 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,037 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,040 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,042 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,044 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,046 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,048 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,050 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,053 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,055 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,058 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,060 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,063 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,065 copying lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,067 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,069 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,071 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,073 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,075 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,077 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,080 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,082 copying lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:28,084 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,086 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,088 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,090 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,092 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,094 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,097 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,099 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,101 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,103 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,106 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,108 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,110 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,112 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,115 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,117 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,119 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,121 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,123 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,125 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,128 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,130 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,132 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,134 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,136 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,139 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,141 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,143 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,145 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,147 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,150 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,152 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,154 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,156 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,159 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,161 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,163 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,165 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,167 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,170 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,172 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,174 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,176 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,178 copying lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,180 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,183 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,185 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,187 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,189 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,192 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,194 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,196 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,198 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,200 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,203 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,205 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,207 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,209 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,212 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,214 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,216 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,219 copying lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:28,221 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,223 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,225 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,228 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,230 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,232 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,234 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,237 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,239 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,241 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,244 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,246 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,248 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,250 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,253 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,255 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,257 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,260 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,262 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,264 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,267 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,269 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,271 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,274 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,276 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,278 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,281 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,283 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,285 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,288 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,290 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,292 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,294 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,296 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,299 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,301 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,303 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,314 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,316 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,319 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,321 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,323 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,325 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,327 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,330 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,332 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,334 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,336 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,339 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,341 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,344 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,346 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,348 copying lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,350 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,352 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,355 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,357 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,359 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,361 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,363 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,365 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,367 copying lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:28,370 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,372 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,374 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,376 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,379 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,381 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,383 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,386 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,388 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,391 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,393 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,396 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,398 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,400 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,403 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,405 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,408 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,410 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,412 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,415 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,417 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,420 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,422 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,424 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,427 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,429 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,431 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,434 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,436 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,438 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,441 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,443 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,445 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,448 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,450 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,453 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,455 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,457 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,459 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,462 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,464 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,466 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,469 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,471 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,474 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,476 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,478 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,481 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,483 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,485 copying lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,488 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,490 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,492 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,495 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,497 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,499 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,501 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,504 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,506 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,508 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,510 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,512 copying lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:28,515 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,517 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,519 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,522 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,524 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,526 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,529 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,531 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,533 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,535 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,537 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,540 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,542 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,544 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,547 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,549 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,551 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,553 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,556 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,558 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,560 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,563 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,565 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,567 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,570 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,572 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,574 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,577 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,579 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,582 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,584 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,586 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,589 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,591 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,593 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,595 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,597 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,600 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,602 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,604 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,606 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,608 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,611 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,613 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,615 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,617 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,620 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,622 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,624 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,627 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,629 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,631 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,634 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,636 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,639 copying lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,641 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,644 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,646 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,649 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,651 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,653 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,655 copying lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:28,657 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,659 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,661 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,664 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,666 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,668 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,670 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,672 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,675 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,677 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,679 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,682 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,684 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,686 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,688 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,690 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,693 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,695 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,697 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,699 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,702 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,704 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,706 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,708 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,710 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,712 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,714 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,717 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,719 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,721 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,723 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,725 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,728 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,730 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,732 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,734 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,737 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,739 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,741 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,744 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,746 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,749 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,751 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,753 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,756 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,758 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,760 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,763 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,765 copying lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,767 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,770 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,772 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,774 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,776 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,779 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,781 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,784 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,787 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,789 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,792 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,794 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,796 copying lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:28,798 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,800 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,802 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,805 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,807 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,809 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,811 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,813 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,816 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,818 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,821 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,823 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,826 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,829 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,831 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,833 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,835 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,838 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,840 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,842 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,844 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,846 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,849 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,851 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,853 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,855 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,857 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,860 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,862 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,864 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,867 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,869 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,871 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,874 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,876 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,878 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,880 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,882 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,885 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,887 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,889 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,891 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,894 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,896 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,898 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,900 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,903 copying lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,905 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,907 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,909 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,911 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,914 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,916 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,918 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,920 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,922 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,924 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,927 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,929 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,931 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,933 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,936 copying lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:28,938 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,940 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,942 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,944 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,946 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,949 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,951 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,953 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,956 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,958 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,960 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,962 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,964 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,967 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,969 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,971 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,973 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,976 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,978 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,980 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,982 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,984 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,987 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,989 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,991 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,994 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,996 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:28,998 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,000 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,003 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,005 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,007 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,009 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,011 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,013 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,015 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,018 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,020 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,022 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,024 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,026 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,029 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,031 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,033 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,036 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,038 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,040 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,042 copying lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,045 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,047 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,050 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,052 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,055 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,057 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,060 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,062 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,064 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,066 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,069 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,071 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,073 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,076 copying lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:29,078 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,080 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,082 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,084 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,087 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,089 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,091 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,094 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,096 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,098 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,100 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,103 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,105 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,107 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,109 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,112 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,114 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,116 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,118 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,120 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,122 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,124 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,127 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,129 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,131 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,133 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,135 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,138 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,140 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,142 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,144 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,146 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,148 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,150 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,153 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,155 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,157 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,159 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,161 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,163 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,165 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,167 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,169 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,172 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,174 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,176 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,178 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,180 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,182 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,184 copying lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,186 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,188 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,190 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,192 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,194 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,196 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,198 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,200 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,202 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,204 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,206 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,208 copying lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:29,210 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,212 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,214 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,217 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,219 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,221 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,223 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,225 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,227 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,229 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,231 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,233 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,235 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,236 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,239 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,241 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,243 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,245 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,247 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,249 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,251 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,253 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,255 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,257 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,259 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,261 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,263 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,266 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,268 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,270 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,272 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,274 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,276 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,278 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,280 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,282 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,284 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,286 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,288 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,290 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,292 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,294 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,296 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,299 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,301 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,303 copying lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,305 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,307 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,310 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,312 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,314 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,316 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,318 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,320 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,322 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,324 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,326 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,328 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,330 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,333 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,334 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,336 copying lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:29,338 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,340 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,342 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,344 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,346 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,348 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,350 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,352 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,354 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,356 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,358 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,360 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,362 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,364 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,366 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,368 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,370 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,372 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,374 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,376 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,378 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,380 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,382 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,384 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,386 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,388 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,390 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,392 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,394 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,395 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,397 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,399 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,401 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,404 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,405 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,407 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,410 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,412 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,414 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,416 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,418 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,420 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,422 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,424 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,426 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,428 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,430 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,432 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,434 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,436 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,438 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,440 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,442 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,444 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,446 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,449 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,451 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,454 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,456 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,459 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,462 copying lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,464 copying lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:29,466 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,469 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,471 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,474 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,477 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,480 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,482 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,485 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,487 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,490 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,492 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,495 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,497 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,500 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,502 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,504 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,507 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,509 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,512 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,514 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,516 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,519 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,521 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,524 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,526 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,529 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,531 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,534 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,537 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,539 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,542 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,544 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,546 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,549 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,551 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,553 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,555 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,557 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,559 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,562 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,564 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,566 copying lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,568 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,570 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,571 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,573 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,575 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,577 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,579 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,581 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,583 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,585 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,587 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,589 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,591 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,593 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,595 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,597 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,599 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,602 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,604 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,606 copying lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:29,608 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,610 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,612 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,614 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,616 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,618 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,620 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,622 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,624 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,626 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,628 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,630 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,632 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,634 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,636 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,638 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,640 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,642 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,644 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,646 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,648 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,650 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,652 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,654 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,656 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,658 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,660 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,663 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,665 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,667 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,669 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,671 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,673 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,676 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,678 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,680 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,682 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,684 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,686 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,688 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,691 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,693 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,695 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,697 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,699 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,701 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,703 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,705 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,707 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,710 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,712 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,714 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,716 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,718 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,720 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,722 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,724 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,726 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,728 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,730 copying lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,732 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,734 copying lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:29,736 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,739 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,741 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,743 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,745 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,747 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,750 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,752 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,754 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,756 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,759 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,761 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,763 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,765 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,767 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,769 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,771 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,773 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,776 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,778 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,780 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,782 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,784 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,786 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,788 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,790 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,792 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,794 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,796 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,799 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,801 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,803 copying lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,806 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,808 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,810 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,812 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,814 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,817 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,819 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,822 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,824 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,826 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,828 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,831 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,833 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,835 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,837 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,840 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,842 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,844 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,846 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,848 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,850 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,852 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,855 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,857 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,859 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,861 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,864 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,866 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,868 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,871 copying lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:29,873 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,875 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,877 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,879 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,881 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,884 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,886 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,888 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,890 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,892 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,894 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,896 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,898 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,900 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,902 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,904 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,907 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,909 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,911 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,913 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,915 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,917 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,919 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,922 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,924 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,926 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,928 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,931 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,933 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,935 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,937 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,939 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,942 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,944 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,946 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,948 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,950 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,952 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,955 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,957 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,959 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,961 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,963 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,965 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,967 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,970 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,972 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,974 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,977 copying lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,979 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,981 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,983 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,986 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,988 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,990 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,992 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,995 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:29,997 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:30,000 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:30,002 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:30,004 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:30,007 copying lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:30,009 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,011 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,013 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,015 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,017 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,019 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,021 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,024 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,026 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,028 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,030 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,032 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,034 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,037 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,039 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,041 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,043 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,045 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,047 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,049 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,051 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,053 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,056 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,058 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,060 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,062 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,064 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,066 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,069 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,071 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,073 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,075 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,079 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,081 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,084 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,086 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,088 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,090 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,092 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,094 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,097 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,099 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,101 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,103 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,105 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,108 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,110 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,112 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,114 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,116 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,119 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,121 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,123 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,125 copying lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,127 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,129 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,131 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,134 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,136 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,139 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,142 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,144 copying lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:30,147 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,149 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,151 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,154 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,156 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,158 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,160 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,162 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,165 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,167 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,169 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,172 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,174 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,176 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,178 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,180 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,183 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,185 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,187 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,189 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,191 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,193 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,196 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,198 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,200 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,202 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,204 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,206 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,208 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,210 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,212 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,215 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,217 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,219 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,221 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,223 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,225 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,227 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,229 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,231 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,233 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,235 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,238 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,239 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,241 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,243 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,245 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,247 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,249 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,251 copying lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,253 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,255 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,257 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,259 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,261 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,264 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,266 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,268 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,270 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,272 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,275 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,277 copying lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:30,279 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,281 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,283 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,286 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,288 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,290 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,292 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,294 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,296 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,299 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,301 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,303 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,305 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,307 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,310 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,312 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,314 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,316 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,318 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,321 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,323 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,325 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,327 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,330 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,332 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,334 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,337 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,339 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,341 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,343 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,346 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,348 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,350 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,352 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,355 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,357 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,359 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,361 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,363 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,365 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,367 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,369 copying lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,371 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,373 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,376 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,378 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,380 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,382 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,384 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,386 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,389 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,391 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,393 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,395 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,398 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,400 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,402 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,405 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,407 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,409 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,411 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,413 copying lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:30,416 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,418 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,420 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,422 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,424 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,426 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,429 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,431 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,433 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,435 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,437 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,439 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,442 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,444 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,446 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,448 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,450 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,453 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,455 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,457 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,459 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,462 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,464 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,466 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,468 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,470 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,472 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,474 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,476 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,479 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,481 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,483 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,485 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,487 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,489 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,492 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,494 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,496 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,498 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,501 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,503 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,505 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,507 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,510 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,512 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,514 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,517 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,519 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,522 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,524 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,526 copying lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,529 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,531 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,533 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,535 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,537 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,540 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,542 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,544 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,546 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,549 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,551 copying lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:30,553 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,555 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,557 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,559 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,562 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,564 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,566 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,569 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,571 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,573 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,575 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,578 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,580 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,582 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,585 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,587 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,589 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,591 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,593 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,595 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,597 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,600 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,602 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,604 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,606 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,608 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,611 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,613 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,615 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,617 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,619 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,622 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,624 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,626 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,628 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,631 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,633 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,635 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,638 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,640 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,642 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,644 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,646 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,648 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,650 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,653 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,655 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,657 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,659 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,662 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,664 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,666 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,668 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,671 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,673 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,675 copying lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,677 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,680 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,682 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,684 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,687 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,689 copying lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:30,691 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,693 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,696 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,698 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,700 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,702 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,704 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,706 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,709 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,711 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,713 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,715 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,717 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,719 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,721 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,723 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,726 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,728 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,730 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,732 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,734 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,736 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,739 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,741 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,743 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,745 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,748 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,750 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,752 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,754 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,756 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,759 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,761 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,763 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,765 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,768 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,770 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,772 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,774 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,776 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,779 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,781 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,783 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,785 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,787 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,789 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,792 copying lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,794 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,796 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,799 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,801 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,803 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,806 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,808 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,810 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,812 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,815 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,817 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,819 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,821 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,823 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,825 copying lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:30,827 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,830 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,832 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,834 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,836 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,838 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,840 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,842 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,845 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,847 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,849 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,851 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,854 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,856 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,858 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,860 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,863 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,865 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,867 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,870 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,872 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,874 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,876 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,878 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,880 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,882 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,884 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,886 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,889 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,891 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,893 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,895 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,897 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,900 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,902 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,904 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,906 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,909 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,911 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,913 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,915 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,918 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,920 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,922 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,924 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,926 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,928 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,931 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,933 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,935 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,937 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,939 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,941 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,943 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,945 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,947 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,950 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,952 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,954 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,956 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,960 copying lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,963 copying lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:30,966 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,969 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,972 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,975 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,978 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,981 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,984 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,987 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,990 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,992 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,995 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:30,998 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,001 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,003 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,006 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,009 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,012 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,015 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,018 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,021 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,024 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,026 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,029 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,032 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,035 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,038 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,041 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,044 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,046 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,049 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,052 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,055 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,057 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,060 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,062 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,064 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,066 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,068 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,070 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,072 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,074 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,077 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,079 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,081 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,083 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,086 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,088 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,090 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,092 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,094 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,096 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,099 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,101 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,103 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,105 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,108 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,110 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,112 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,115 copying lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,117 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,119 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,121 copying lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:31,124 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,126 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,128 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,130 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,132 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,134 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,137 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,139 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,141 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,143 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,145 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,148 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,150 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,152 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,154 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,156 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,159 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,161 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,163 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,165 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,167 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,170 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,172 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,174 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,176 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,179 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,181 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,183 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,186 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,188 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,191 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,193 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,195 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,198 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,200 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,202 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,204 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,207 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,209 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,211 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,213 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,215 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,218 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,220 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,223 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,225 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,229 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,232 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,234 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,236 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,239 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,241 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,243 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,246 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,248 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,250 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,252 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,255 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,257 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,259 copying lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,262 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,264 copying lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:31,267 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,270 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_abstract_algebra.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,273 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_aging.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,276 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_business_ethics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,279 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_geography.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,281 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,283 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_anatomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,286 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,288 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_world_religions.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,290 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_miscellaneous.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,293 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,295 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,297 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_public_relations.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,300 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_world_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,302 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_sexuality.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,304 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,306 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_european_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,309 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_machine_learning.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,311 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_global_facts.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,314 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_medical_genetics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,316 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_statistics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,318 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_sociology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,321 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_scenarios.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,323 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_security_studies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,325 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_biology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,327 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_formal_logic.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,330 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_prehistory.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,332 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_international_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,334 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_virology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,337 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_computer_security.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,339 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_electrical_engineering.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,341 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_disputes.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,343 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_us_history.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,346 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_logical_fallacies.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,348 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_stem.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,351 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,353 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_econometrics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,355 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_law.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,358 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_astronomy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,360 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_conceptual_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,363 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,365 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_humanities.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,367 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,370 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_social_sciences.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,372 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_management.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,374 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,376 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_accounting.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,379 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,381 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_marketing.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,383 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_psychology.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,386 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_other.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,388 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,390 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_jurisprudence.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,393 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,395 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_philosophy.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,398 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_nutrition.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,400 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_medicine.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,403 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,405 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,407 copying lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,410 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,412 copying lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_physics.yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:31,415 copying lm_eval/tasks/tmlu/default/tmlu_nutritionist.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,417 copying lm_eval/tasks/tmlu/default/tmlu_lawyer_qualification.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,419 copying lm_eval/tasks/tmlu/default/tmlu_tour_guide.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,421 copying lm_eval/tasks/tmlu/default/tmlu_AST_history.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,424 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_chinese.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,426 copying lm_eval/tasks/tmlu/default/tmlu_basic_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,428 copying lm_eval/tasks/tmlu/default/_tmlu.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,430 copying lm_eval/tasks/tmlu/default/tmlu_driving_rule.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,432 copying lm_eval/tasks/tmlu/default/tmlu_accountant.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,435 copying lm_eval/tasks/tmlu/default/tmlu_tour_leader.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,437 copying lm_eval/tasks/tmlu/default/tmlu_AST_chemistry.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,439 copying lm_eval/tasks/tmlu/default/tmlu_CAP_chemistry.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,441 copying lm_eval/tasks/tmlu/default/tmlu_CAP_chinese.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,444 copying lm_eval/tasks/tmlu/default/tmlu_AST_biology.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,446 copying lm_eval/tasks/tmlu/default/tmlu_AST_civics.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,448 copying lm_eval/tasks/tmlu/default/tmlu_CAP_geography.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,450 copying lm_eval/tasks/tmlu/default/tmlu_CAP_civics.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,453 copying lm_eval/tasks/tmlu/default/tmlu_clinical_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,455 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_civics.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,457 copying lm_eval/tasks/tmlu/default/tmlu_taiwan_tourist_resources.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,459 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_geography.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,461 copying lm_eval/tasks/tmlu/default/tmlu_teacher_qualification.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,463 copying lm_eval/tasks/tmlu/default/tmlu_CAP_biology.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,466 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_chemistry.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,468 copying lm_eval/tasks/tmlu/default/tmlu_clinical_psychologist.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,471 copying lm_eval/tasks/tmlu/default/tmlu_CAP_history.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,473 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_earth_science.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,475 copying lm_eval/tasks/tmlu/default/tmlu_AST_chinese.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,477 copying lm_eval/tasks/tmlu/default/tmlu_CAP_earth_science.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,479 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_biology.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,482 copying lm_eval/tasks/tmlu/default/tmlu_AST_geography.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,484 copying lm_eval/tasks/tmlu/default/tmlu_GSAT_history.yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:31,486 copying lm_eval/tasks/drop/default.yaml -> build/lib/lm_eval/tasks/drop 2026-05-11T15:52:31,489 creating build/lib/lm_eval/tasks/gsm_plus 2026-05-11T15:52:31,490 copying lm_eval/tasks/gsm_plus/gsm_plus_mini.yaml -> build/lib/lm_eval/tasks/gsm_plus 2026-05-11T15:52:31,492 copying lm_eval/tasks/gsm_plus/gsm_plus.yaml -> build/lib/lm_eval/tasks/gsm_plus 2026-05-11T15:52:31,495 copying lm_eval/tasks/cmmlu/cmmlu_chinese_food_culture.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,497 copying lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,500 copying lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,502 copying lm_eval/tasks/cmmlu/cmmlu_journalism.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,504 copying lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,506 copying lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,509 copying lm_eval/tasks/cmmlu/cmmlu_agronomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,511 copying lm_eval/tasks/cmmlu/cmmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,513 copying lm_eval/tasks/cmmlu/cmmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,515 copying lm_eval/tasks/cmmlu/cmmlu_chinese_history.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,517 copying lm_eval/tasks/cmmlu/cmmlu_legal_and_moral_basis.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,520 copying lm_eval/tasks/cmmlu/cmmlu_food_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,522 copying lm_eval/tasks/cmmlu/cmmlu_international_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,524 copying lm_eval/tasks/cmmlu/cmmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,527 copying lm_eval/tasks/cmmlu/cmmlu_chinese_literature.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,529 copying lm_eval/tasks/cmmlu/cmmlu_global_facts.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,531 copying lm_eval/tasks/cmmlu/cmmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,534 copying lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,536 copying lm_eval/tasks/cmmlu/cmmlu_management.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,538 copying lm_eval/tasks/cmmlu/cmmlu_arts.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,541 copying lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,543 copying lm_eval/tasks/cmmlu/cmmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,545 copying lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,548 copying lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,550 copying lm_eval/tasks/cmmlu/cmmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,552 copying lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,555 copying lm_eval/tasks/cmmlu/cmmlu_ancient_chinese.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,557 copying lm_eval/tasks/cmmlu/cmmlu_chinese_driving_rule.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,560 copying lm_eval/tasks/cmmlu/cmmlu_logical.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,562 copying lm_eval/tasks/cmmlu/cmmlu_virology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,565 copying lm_eval/tasks/cmmlu/_cmmlu.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,567 copying lm_eval/tasks/cmmlu/cmmlu_chinese_foreign_policy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,569 copying lm_eval/tasks/cmmlu/cmmlu_elementary_information_and_technology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,572 copying lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,574 copying lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,577 copying lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,579 copying lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,582 copying lm_eval/tasks/cmmlu/cmmlu_genetics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,584 copying lm_eval/tasks/cmmlu/cmmlu_anatomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,586 copying lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,589 copying lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,591 copying lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,593 copying lm_eval/tasks/cmmlu/cmmlu_world_history.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,595 copying lm_eval/tasks/cmmlu/cmmlu_construction_project_management.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,598 copying lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,600 copying lm_eval/tasks/cmmlu/cmmlu_college_education.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,602 copying lm_eval/tasks/cmmlu/cmmlu_chinese_civil_service_exam.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,604 copying lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,607 copying lm_eval/tasks/cmmlu/cmmlu_astronomy.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,609 copying lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,612 copying lm_eval/tasks/cmmlu/cmmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,614 copying lm_eval/tasks/cmmlu/cmmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,616 copying lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,619 copying lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,621 copying lm_eval/tasks/cmmlu/cmmlu_chinese_teacher_qualification.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,623 copying lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,625 copying lm_eval/tasks/cmmlu/cmmlu_economics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,628 copying lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,630 copying lm_eval/tasks/cmmlu/cmmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,632 copying lm_eval/tasks/cmmlu/cmmlu_sociology.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,635 copying lm_eval/tasks/cmmlu/cmmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,637 copying lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,639 copying lm_eval/tasks/cmmlu/cmmlu_education.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,642 copying lm_eval/tasks/cmmlu/cmmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,644 copying lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,647 copying lm_eval/tasks/cmmlu/cmmlu_security_study.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,649 copying lm_eval/tasks/cmmlu/cmmlu_college_law.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,651 copying lm_eval/tasks/cmmlu/cmmlu_marketing.yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:31,654 copying lm_eval/tasks/logiqa/logiqa.yaml -> build/lib/lm_eval/tasks/logiqa 2026-05-11T15:52:31,656 copying lm_eval/tasks/race/race.yaml -> build/lib/lm_eval/tasks/race 2026-05-11T15:52:31,659 copying lm_eval/tasks/spanish_bench/spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,661 copying lm_eval/tasks/spanish_bench/wnli_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,664 copying lm_eval/tasks/spanish_bench/cocoteros_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,666 copying lm_eval/tasks/spanish_bench/mgsm_direct_es_spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,668 copying lm_eval/tasks/spanish_bench/copa_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,670 copying lm_eval/tasks/spanish_bench/escola.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,673 copying lm_eval/tasks/spanish_bench/paws_es_spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,675 copying lm_eval/tasks/spanish_bench/openbookqa_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,677 copying lm_eval/tasks/spanish_bench/xlsum_es.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,679 copying lm_eval/tasks/spanish_bench/xnli_es_spanish_bench.yaml -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:31,681 creating build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:52:31,682 copying lm_eval/tasks/spanish_bench/phrases_es/phrases_va-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:52:31,684 copying lm_eval/tasks/spanish_bench/phrases_es/phrases_es-va.yaml -> build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:52:31,687 copying lm_eval/tasks/spanish_bench/flores_es/flores_pt-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,689 copying lm_eval/tasks/spanish_bench/flores_es/flores_es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,691 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-eu.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,693 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-en.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,695 copying lm_eval/tasks/spanish_bench/flores_es/flores_it-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,697 copying lm_eval/tasks/spanish_bench/flores_es/flores_eu-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,700 copying lm_eval/tasks/spanish_bench/flores_es/flores_gl-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,702 copying lm_eval/tasks/spanish_bench/flores_es/flores_en-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,704 copying lm_eval/tasks/spanish_bench/flores_es/flores_ca-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,706 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-pt.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,708 copying lm_eval/tasks/spanish_bench/flores_es/flores_de-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,710 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-fr.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,713 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-de.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,715 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-gl.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,717 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-ca.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,719 copying lm_eval/tasks/spanish_bench/flores_es/flores_es-it.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,722 copying lm_eval/tasks/spanish_bench/flores_es/flores_fr-es.yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:31,724 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:31,727 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:31,729 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:31,732 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:31,734 copying lm_eval/tasks/include/few_shot_en/Indonesian/_include_base_44_indonesian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:31,736 copying lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:31,739 copying lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:31,741 copying lm_eval/tasks/include/few_shot_en/Bulgarian/_include_base_44_bulgarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:31,743 copying lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:31,746 copying lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:31,750 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:31,752 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:31,755 copying lm_eval/tasks/include/few_shot_en/Bengali/_include_base_44_bengali.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:31,757 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:31,759 copying lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:31,762 copying lm_eval/tasks/include/few_shot_en/Turkish/_include_base_44_turkish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:31,764 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:31,766 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:31,769 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:31,771 copying lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:31,774 copying lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:31,776 copying lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:31,778 copying lm_eval/tasks/include/few_shot_en/Belarusian/_include_base_44_belarusian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:31,781 copying lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:31,783 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,786 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,788 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,791 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,793 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,795 copying lm_eval/tasks/include/few_shot_en/Greek/_include_base_44_greek.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,798 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,800 copying lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:31,802 copying lm_eval/tasks/include/few_shot_en/Hungarian/_include_base_44_hungarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:31,805 copying lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:31,807 copying lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:31,809 copying lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:31,812 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:31,814 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:31,817 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:31,819 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:31,821 copying lm_eval/tasks/include/few_shot_en/Dutch/_include_base_44_dutch.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:31,824 copying lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:31,826 copying lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:52:31,829 copying lm_eval/tasks/include/few_shot_en/Korean/_include_base_44_korean.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:52:31,831 copying lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:52:31,833 copying lm_eval/tasks/include/few_shot_en/German/include_base_44_german_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:31,836 copying lm_eval/tasks/include/few_shot_en/German/_include_base_44_german.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:31,838 copying lm_eval/tasks/include/few_shot_en/German/include_base_44_german_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:31,840 copying lm_eval/tasks/include/few_shot_en/German/include_base_44_german_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:31,843 copying lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:31,845 copying lm_eval/tasks/include/few_shot_en/Polish/_include_base_44_polish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:31,848 copying lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:31,850 copying lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:31,853 copying lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:31,855 copying lm_eval/tasks/include/few_shot_en/Urdu/_include_base_44_urdu.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:31,857 copying lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:31,859 copying lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:31,862 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:31,864 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:31,866 copying lm_eval/tasks/include/few_shot_en/North Macedonian/_include_base_44_north macedonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:31,869 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:31,871 copying lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:31,874 copying lm_eval/tasks/include/few_shot_en/Ukrainian/_include_base_44_ukrainian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:31,876 copying lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:31,879 copying lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:31,881 copying lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:31,883 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,885 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,888 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,890 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,893 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,895 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,898 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,900 copying lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,903 copying lm_eval/tasks/include/few_shot_en/Chinese/_include_base_44_chinese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:31,905 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,907 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,910 copying lm_eval/tasks/include/few_shot_en/Portuguese/_include_base_44_portuguese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,912 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,914 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,917 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,919 copying lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:31,922 copying lm_eval/tasks/include/few_shot_en/Georgian/_include_base_44_georgian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:52:31,924 copying lm_eval/tasks/include/few_shot_en/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:52:31,926 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:31,929 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:31,931 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:31,934 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:31,936 copying lm_eval/tasks/include/few_shot_en/French/include_base_44_french_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:31,938 copying lm_eval/tasks/include/few_shot_en/French/_include_base_44_french.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:31,941 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:31,943 copying lm_eval/tasks/include/few_shot_en/Spanish/_include_base_44_spanish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:31,946 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:31,948 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:31,950 copying lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:31,953 copying lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:31,955 copying lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:31,957 copying lm_eval/tasks/include/few_shot_en/Serbian/_include_base_44_serbian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:31,960 copying lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:31,962 copying lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:31,964 copying lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:31,966 copying lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:31,969 copying lm_eval/tasks/include/few_shot_en/Malay/_include_base_44_malay.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:31,971 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,973 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,975 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,978 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,980 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,983 copying lm_eval/tasks/include/few_shot_en/Arabic/_include_base_44_arabic.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,985 copying lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:31,987 copying lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:31,990 copying lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:31,992 copying lm_eval/tasks/include/few_shot_en/Vietnamese/_include_base_44_vietnamese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:31,994 copying lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:31,997 copying lm_eval/tasks/include/few_shot_en/Russian/_include_base_44_russian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:31,999 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,002 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,004 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,006 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,009 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,011 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,014 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,016 copying lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:32,019 copying lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:32,021 copying lm_eval/tasks/include/few_shot_en/Japanese/_include_base_44_japanese.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:32,023 copying lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:32,026 copying lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:32,028 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,030 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,033 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,035 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,038 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,040 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,042 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:32,045 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:32,047 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:32,050 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:32,052 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:32,054 copying lm_eval/tasks/include/few_shot_en/Persian/_include_base_44_persian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:32,057 copying lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:32,060 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:32,062 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:32,065 copying lm_eval/tasks/include/few_shot_en/Albanian/_include_base_44_albanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:32,067 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:32,070 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:32,072 copying lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:32,075 copying lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:32,077 copying lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:32,079 copying lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:32,082 copying lm_eval/tasks/include/few_shot_en/Croatian/_include_base_44_croatian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:32,084 copying lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:52:32,087 copying lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:52:32,089 copying lm_eval/tasks/include/few_shot_en/Nepali/_include_base_44_nepali.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:52:32,092 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,094 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,097 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,099 copying lm_eval/tasks/include/few_shot_en/Italian/_include_base_44_italian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,101 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,104 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,106 copying lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:32,109 copying lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:52:32,111 copying lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:52:32,114 copying lm_eval/tasks/include/few_shot_en/Tamil/_include_base_44_tamil.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:52:32,116 copying lm_eval/tasks/include/few_shot_en/Basque/include_base_44_basque_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:52:32,119 copying lm_eval/tasks/include/few_shot_en/Basque/_include_base_44_basque.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:52:32,122 copying lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:52:32,124 copying lm_eval/tasks/include/few_shot_en/Hebrew/_include_base_44_hebrew.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:52:32,127 copying lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:52:32,129 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:32,132 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:32,134 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:32,136 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:32,139 copying lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:32,141 copying lm_eval/tasks/include/few_shot_en/Estonian/_include_base_44_estonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:32,143 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:32,146 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:32,148 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:32,150 copying lm_eval/tasks/include/few_shot_en/Telugu/_include_base_44_telugu.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:32,153 copying lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:32,155 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,158 copying lm_eval/tasks/include/few_shot_en/Malayalam/_include_base_44_malayalam.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,160 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,163 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,165 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,167 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,170 copying lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:32,173 copying lm_eval/tasks/include/few_shot_en/Kazakh/_include_base_44_kazakh.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:52:32,175 copying lm_eval/tasks/include/few_shot_en/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:52:32,178 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:32,181 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:32,183 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:32,186 copying lm_eval/tasks/include/few_shot_en/Uzbek/_include_base_44_uzbek.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:32,188 copying lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:32,191 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:32,193 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:32,196 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:32,198 copying lm_eval/tasks/include/few_shot_en/Armenian/_include_base_44_armenian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:32,200 copying lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:32,203 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:32,205 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:32,208 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:32,210 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:32,213 copying lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:32,215 copying lm_eval/tasks/include/few_shot_en/Finnish/_include_base_44_finnish.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:32,218 copying lm_eval/tasks/include/few_shot_en/Tagalog/_include_base_44_tagalog.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:52:32,220 copying lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:52:32,223 copying lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:52:32,225 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:32,228 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:32,230 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:32,233 copying lm_eval/tasks/include/few_shot_en/Lithuanian/_include_base_44_lithuanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:32,235 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:32,238 copying lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:32,241 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,243 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,246 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,248 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,250 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,253 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,255 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,258 copying lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,260 copying lm_eval/tasks/include/few_shot_en/Hindi/_include_base_44_hindi.yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:32,262 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:32,265 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:32,267 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:32,270 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:32,272 copying lm_eval/tasks/include/few_shot_og/Indonesian/_include_base_44_indonesian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:32,275 copying lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:32,277 copying lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:32,280 copying lm_eval/tasks/include/few_shot_og/Bulgarian/_include_base_44_bulgarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:32,282 copying lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:32,285 copying lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:32,287 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:32,290 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:32,292 copying lm_eval/tasks/include/few_shot_og/Bengali/_include_base_44_bengali.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:32,295 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:32,297 copying lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:32,300 copying lm_eval/tasks/include/few_shot_og/Turkish/_include_base_44_turkish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:32,303 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:32,305 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:32,308 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:32,310 copying lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:32,313 copying lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:32,315 copying lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:32,317 copying lm_eval/tasks/include/few_shot_og/Belarusian/_include_base_44_belarusian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:32,320 copying lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:32,322 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,325 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,327 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,330 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,332 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,335 copying lm_eval/tasks/include/few_shot_og/Greek/_include_base_44_greek.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,337 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,340 copying lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:32,342 copying lm_eval/tasks/include/few_shot_og/Hungarian/_include_base_44_hungarian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:32,345 copying lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:32,347 copying lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:32,350 copying lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:32,353 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:32,355 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:32,358 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:32,361 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:32,363 copying lm_eval/tasks/include/few_shot_og/Dutch/_include_base_44_dutch.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:32,366 copying lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:32,368 copying lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:52:32,371 copying lm_eval/tasks/include/few_shot_og/Korean/_include_base_44_korean.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:52:32,373 copying lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:52:32,376 copying lm_eval/tasks/include/few_shot_og/German/include_base_44_german_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:32,378 copying lm_eval/tasks/include/few_shot_og/German/_include_base_44_german.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:32,381 copying lm_eval/tasks/include/few_shot_og/German/include_base_44_german_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:32,383 copying lm_eval/tasks/include/few_shot_og/German/include_base_44_german_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:32,386 copying lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:32,390 copying lm_eval/tasks/include/few_shot_og/Polish/_include_base_44_polish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:32,392 copying lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:32,395 copying lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:32,397 copying lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:32,400 copying lm_eval/tasks/include/few_shot_og/Urdu/_include_base_44_urdu.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:32,402 copying lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:32,405 copying lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:32,407 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:32,410 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:32,412 copying lm_eval/tasks/include/few_shot_og/North Macedonian/_include_base_44_north macedonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:32,415 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:32,417 copying lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:32,420 copying lm_eval/tasks/include/few_shot_og/Ukrainian/_include_base_44_ukrainian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:32,423 copying lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:32,426 copying lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:32,429 copying lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:32,432 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,434 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,437 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,439 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,442 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,444 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,446 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,449 copying lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,451 copying lm_eval/tasks/include/few_shot_og/Chinese/_include_base_44_chinese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:32,454 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,456 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,459 copying lm_eval/tasks/include/few_shot_og/Portuguese/_include_base_44_portuguese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,461 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,464 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,467 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,470 copying lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:32,473 copying lm_eval/tasks/include/few_shot_og/Georgian/_include_base_44_georgian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:52:32,476 copying lm_eval/tasks/include/few_shot_og/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:52:32,479 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:32,481 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:32,484 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:32,486 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:32,488 copying lm_eval/tasks/include/few_shot_og/French/include_base_44_french_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:32,491 copying lm_eval/tasks/include/few_shot_og/French/_include_base_44_french.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:32,493 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:32,496 copying lm_eval/tasks/include/few_shot_og/Spanish/_include_base_44_spanish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:32,498 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:32,500 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:32,503 copying lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:32,505 copying lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:32,507 copying lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:32,510 copying lm_eval/tasks/include/few_shot_og/Serbian/_include_base_44_serbian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:32,512 copying lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:32,515 copying lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:32,517 copying lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:32,520 copying lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:32,522 copying lm_eval/tasks/include/few_shot_og/Malay/_include_base_44_malay.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:32,525 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,527 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,530 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,532 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,535 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,537 copying lm_eval/tasks/include/few_shot_og/Arabic/_include_base_44_arabic.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,540 copying lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:32,542 copying lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:32,545 copying lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:32,547 copying lm_eval/tasks/include/few_shot_og/Vietnamese/_include_base_44_vietnamese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:32,550 copying lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:32,552 copying lm_eval/tasks/include/few_shot_og/Russian/_include_base_44_russian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,555 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,557 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,559 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,562 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,564 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,566 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,569 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,571 copying lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:32,574 copying lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:32,576 copying lm_eval/tasks/include/few_shot_og/Japanese/_include_base_44_japanese.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:32,579 copying lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:32,581 copying lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:32,584 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,586 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,589 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,591 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,594 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,596 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,599 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:32,601 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:32,603 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:32,606 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:32,608 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:32,611 copying lm_eval/tasks/include/few_shot_og/Persian/_include_base_44_persian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:32,613 copying lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:32,616 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:32,618 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:32,620 copying lm_eval/tasks/include/few_shot_og/Albanian/_include_base_44_albanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:32,623 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:32,625 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:32,628 copying lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:32,630 copying lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:32,633 copying lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:32,635 copying lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:32,638 copying lm_eval/tasks/include/few_shot_og/Croatian/_include_base_44_croatian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:32,640 copying lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:52:32,643 copying lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:52:32,646 copying lm_eval/tasks/include/few_shot_og/Nepali/_include_base_44_nepali.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:52:32,648 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,651 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,653 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,656 copying lm_eval/tasks/include/few_shot_og/Italian/_include_base_44_italian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,658 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,661 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,663 copying lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:32,665 copying lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:52:32,668 copying lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:52:32,670 copying lm_eval/tasks/include/few_shot_og/Tamil/_include_base_44_tamil.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:52:32,673 copying lm_eval/tasks/include/few_shot_og/Basque/include_base_44_basque_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:52:32,675 copying lm_eval/tasks/include/few_shot_og/Basque/_include_base_44_basque.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:52:32,678 copying lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:52:32,680 copying lm_eval/tasks/include/few_shot_og/Hebrew/_include_base_44_hebrew.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:52:32,682 copying lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:52:32,685 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:32,687 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:32,689 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:32,692 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:32,694 copying lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:32,696 copying lm_eval/tasks/include/few_shot_og/Estonian/_include_base_44_estonian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:32,698 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:32,700 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:32,703 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:32,705 copying lm_eval/tasks/include/few_shot_og/Telugu/_include_base_44_telugu.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:32,707 copying lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:32,709 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,712 copying lm_eval/tasks/include/few_shot_og/Malayalam/_include_base_44_malayalam.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,714 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,716 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,718 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,721 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,723 copying lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:32,725 copying lm_eval/tasks/include/few_shot_og/Kazakh/_include_base_44_kazakh.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:52:32,727 copying lm_eval/tasks/include/few_shot_og/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:52:32,730 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:32,732 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:32,734 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:32,736 copying lm_eval/tasks/include/few_shot_og/Uzbek/_include_base_44_uzbek.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:32,738 copying lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:32,741 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:32,743 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:32,745 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:32,747 copying lm_eval/tasks/include/few_shot_og/Armenian/_include_base_44_armenian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:32,750 copying lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:32,752 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:32,754 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:32,756 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:32,759 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:32,761 copying lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:32,763 copying lm_eval/tasks/include/few_shot_og/Finnish/_include_base_44_finnish.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:32,765 copying lm_eval/tasks/include/few_shot_og/Tagalog/_include_base_44_tagalog.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:52:32,768 copying lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:52:32,770 copying lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:52:32,772 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:32,774 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:32,776 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:32,778 copying lm_eval/tasks/include/few_shot_og/Lithuanian/_include_base_44_lithuanian.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:32,780 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:32,783 copying lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:32,785 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_stem.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,787 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,789 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,791 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_professional_certification.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,794 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_social_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,796 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_driving_license.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,798 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,800 copying lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_applied_science.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,803 copying lm_eval/tasks/include/few_shot_og/Hindi/_include_base_44_hindi.yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:32,805 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:32,807 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:32,809 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:32,811 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:32,814 copying lm_eval/tasks/include/default/Indonesian/_include_base_44_indonesian.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:32,816 copying lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:32,818 copying lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:32,821 copying lm_eval/tasks/include/default/Bulgarian/_include_base_44_bulgarian.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:32,823 copying lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:32,825 copying lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:32,828 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:32,830 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_stem.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:32,833 copying lm_eval/tasks/include/default/Bengali/_include_base_44_bengali.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:32,835 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:32,838 copying lm_eval/tasks/include/default/Bengali/include_base_44_bengali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:32,840 copying lm_eval/tasks/include/default/Turkish/_include_base_44_turkish.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:32,843 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:32,845 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:32,848 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:32,850 copying lm_eval/tasks/include/default/Turkish/include_base_44_turkish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:32,853 copying lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:32,855 copying lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:32,857 copying lm_eval/tasks/include/default/Belarusian/_include_base_44_belarusian.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:32,860 copying lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:32,862 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,865 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,867 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_stem.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,870 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,872 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,874 copying lm_eval/tasks/include/default/Greek/_include_base_44_greek.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,877 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,879 copying lm_eval/tasks/include/default/Greek/include_base_44_greek_medical_license.yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:32,882 copying lm_eval/tasks/include/default/Hungarian/_include_base_44_hungarian.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:32,885 copying lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:32,887 copying lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:32,889 copying lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:32,892 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:32,894 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:32,897 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:32,899 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_stem.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:32,902 copying lm_eval/tasks/include/default/Dutch/_include_base_44_dutch.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:32,904 copying lm_eval/tasks/include/default/Dutch/include_base_44_dutch_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:32,906 copying lm_eval/tasks/include/default/Korean/include_base_44_korean_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T15:52:32,909 copying lm_eval/tasks/include/default/Korean/_include_base_44_korean.yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T15:52:32,911 copying lm_eval/tasks/include/default/Korean/include_base_44_korean_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T15:52:32,914 copying lm_eval/tasks/include/default/German/include_base_44_german_social_science.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:32,916 copying lm_eval/tasks/include/default/German/_include_base_44_german.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:32,919 copying lm_eval/tasks/include/default/German/include_base_44_german_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:32,921 copying lm_eval/tasks/include/default/German/include_base_44_german_stem.yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:32,923 copying lm_eval/tasks/include/default/Polish/include_base_44_polish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:32,926 copying lm_eval/tasks/include/default/Polish/_include_base_44_polish.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:32,929 copying lm_eval/tasks/include/default/Polish/include_base_44_polish_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:32,931 copying lm_eval/tasks/include/default/Polish/include_base_44_polish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:32,934 copying lm_eval/tasks/include/default/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:32,936 copying lm_eval/tasks/include/default/Urdu/_include_base_44_urdu.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:32,939 copying lm_eval/tasks/include/default/Urdu/include_base_44_urdu_stem.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:32,941 copying lm_eval/tasks/include/default/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:32,944 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:32,947 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:32,949 copying lm_eval/tasks/include/default/North Macedonian/_include_base_44_north macedonian.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:32,951 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:32,954 copying lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:32,956 copying lm_eval/tasks/include/default/Ukrainian/_include_base_44_ukrainian.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:32,959 copying lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:32,961 copying lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:32,964 copying lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:32,966 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,969 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,971 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,974 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,976 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,979 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_stem.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,981 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,984 copying lm_eval/tasks/include/default/Chinese/include_base_44_chinese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,986 copying lm_eval/tasks/include/default/Chinese/_include_base_44_chinese.yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:32,989 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:32,991 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:32,993 copying lm_eval/tasks/include/default/Portuguese/_include_base_44_portuguese.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:32,996 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:32,998 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:33,001 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:33,004 copying lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_stem.yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:33,006 copying lm_eval/tasks/include/default/Georgian/_include_base_44_georgian.yaml -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T15:52:33,009 copying lm_eval/tasks/include/default/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T15:52:33,011 copying lm_eval/tasks/include/default/French/include_base_44_french_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:33,014 copying lm_eval/tasks/include/default/French/include_base_44_french_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:33,016 copying lm_eval/tasks/include/default/French/include_base_44_french_social_science.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:33,018 copying lm_eval/tasks/include/default/French/include_base_44_french_stem.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:33,021 copying lm_eval/tasks/include/default/French/include_base_44_french_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:33,023 copying lm_eval/tasks/include/default/French/_include_base_44_french.yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:33,026 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:33,029 copying lm_eval/tasks/include/default/Spanish/_include_base_44_spanish.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:33,031 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:33,033 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:33,036 copying lm_eval/tasks/include/default/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:33,038 copying lm_eval/tasks/include/default/Serbian/include_base_44_serbian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:33,041 copying lm_eval/tasks/include/default/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:33,043 copying lm_eval/tasks/include/default/Serbian/_include_base_44_serbian.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:33,046 copying lm_eval/tasks/include/default/Serbian/include_base_44_serbian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:33,048 copying lm_eval/tasks/include/default/Malay/include_base_44_malay_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:33,051 copying lm_eval/tasks/include/default/Malay/include_base_44_malay_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:33,053 copying lm_eval/tasks/include/default/Malay/include_base_44_malay_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:33,055 copying lm_eval/tasks/include/default/Malay/_include_base_44_malay.yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:33,058 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,061 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,063 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,066 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,068 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_stem.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,070 copying lm_eval/tasks/include/default/Arabic/_include_base_44_arabic.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,073 copying lm_eval/tasks/include/default/Arabic/include_base_44_arabic_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:33,075 copying lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:33,078 copying lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:33,080 copying lm_eval/tasks/include/default/Vietnamese/_include_base_44_vietnamese.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:33,082 copying lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:33,085 copying lm_eval/tasks/include/default/Russian/_include_base_44_russian.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,087 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,090 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,092 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,095 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,097 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,100 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,102 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,105 copying lm_eval/tasks/include/default/Russian/include_base_44_russian_marine_license.yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:33,107 copying lm_eval/tasks/include/default/Japanese/include_base_44_japanese_medical_license.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:33,110 copying lm_eval/tasks/include/default/Japanese/_include_base_44_japanese.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:33,112 copying lm_eval/tasks/include/default/Japanese/include_base_44_japanese_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:33,114 copying lm_eval/tasks/include/default/Japanese/include_base_44_japanese_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:33,117 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,119 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,122 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,124 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,127 copying lm_eval/tasks/include/default/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,129 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,132 copying lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:33,134 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:33,137 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:33,139 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:33,142 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:33,144 copying lm_eval/tasks/include/default/Persian/_include_base_44_persian.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:33,146 copying lm_eval/tasks/include/default/Persian/include_base_44_persian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:33,149 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:33,151 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:33,154 copying lm_eval/tasks/include/default/Albanian/_include_base_44_albanian.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:33,156 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:33,159 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:33,161 copying lm_eval/tasks/include/default/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:33,164 copying lm_eval/tasks/include/default/Croatian/include_base_44_croatian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:33,166 copying lm_eval/tasks/include/default/Croatian/include_base_44_croatian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:33,169 copying lm_eval/tasks/include/default/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:33,171 copying lm_eval/tasks/include/default/Croatian/_include_base_44_croatian.yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:33,174 copying lm_eval/tasks/include/default/Nepali/include_base_44_nepali_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T15:52:33,177 copying lm_eval/tasks/include/default/Nepali/include_base_44_nepali_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T15:52:33,179 copying lm_eval/tasks/include/default/Nepali/_include_base_44_nepali.yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T15:52:33,182 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,185 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,187 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,189 copying lm_eval/tasks/include/default/Italian/_include_base_44_italian.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,192 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,194 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,197 copying lm_eval/tasks/include/default/Italian/include_base_44_italian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:33,199 copying lm_eval/tasks/include/default/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T15:52:33,202 copying lm_eval/tasks/include/default/Tamil/include_base_44_tamil_stem.yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T15:52:33,205 copying lm_eval/tasks/include/default/Tamil/_include_base_44_tamil.yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T15:52:33,207 copying lm_eval/tasks/include/default/Basque/include_base_44_basque_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T15:52:33,210 copying lm_eval/tasks/include/default/Basque/_include_base_44_basque.yaml -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T15:52:33,212 copying lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:52:33,215 copying lm_eval/tasks/include/default/Hebrew/_include_base_44_hebrew.yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:52:33,217 copying lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:52:33,220 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:33,222 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:33,225 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:33,227 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:33,230 copying lm_eval/tasks/include/default/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:33,232 copying lm_eval/tasks/include/default/Estonian/_include_base_44_estonian.yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:33,235 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:33,238 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_stem.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:33,240 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:33,243 copying lm_eval/tasks/include/default/Telugu/_include_base_44_telugu.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:33,245 copying lm_eval/tasks/include/default/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:33,248 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,250 copying lm_eval/tasks/include/default/Malayalam/_include_base_44_malayalam.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,252 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,255 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_stem.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,257 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,260 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,262 copying lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:33,264 copying lm_eval/tasks/include/default/Kazakh/_include_base_44_kazakh.yaml -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T15:52:33,267 copying lm_eval/tasks/include/default/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T15:52:33,270 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:33,272 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:33,274 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:33,277 copying lm_eval/tasks/include/default/Uzbek/_include_base_44_uzbek.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:33,279 copying lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_stem.yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:33,282 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:33,284 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:33,286 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:33,289 copying lm_eval/tasks/include/default/Armenian/_include_base_44_armenian.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:33,291 copying lm_eval/tasks/include/default/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:33,294 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:33,296 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_stem.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:33,299 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:33,303 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:33,308 copying lm_eval/tasks/include/default/Finnish/include_base_44_finnish_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:33,312 copying lm_eval/tasks/include/default/Finnish/_include_base_44_finnish.yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:33,315 copying lm_eval/tasks/include/default/Tagalog/_include_base_44_tagalog.yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:52:33,318 copying lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:52:33,320 copying lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:52:33,322 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:33,325 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:33,327 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:33,329 copying lm_eval/tasks/include/default/Lithuanian/_include_base_44_lithuanian.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:33,332 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:33,334 copying lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:33,337 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_stem.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,339 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,342 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,344 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_professional_certification.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,347 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_social_science.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,349 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_driving_license.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,352 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,354 copying lm_eval/tasks/include/default/Hindi/include_base_44_hindi_applied_science.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,357 copying lm_eval/tasks/include/default/Hindi/_include_base_44_hindi.yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:33,360 copying lm_eval/tasks/slr_bench/slr_bench_group.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:33,362 copying lm_eval/tasks/slr_bench/slr_bench_all.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:33,364 copying lm_eval/tasks/slr_bench/slr_bench_hard.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:33,366 copying lm_eval/tasks/slr_bench/slr_bench_basic.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:33,369 copying lm_eval/tasks/slr_bench/slr_bench_easy.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:33,371 copying lm_eval/tasks/slr_bench/slr_bench_medium.yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:33,374 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_es.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,376 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_eu.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,379 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_es.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,382 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_gl.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,384 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_gl.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,387 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_eu.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,389 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_gl.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,391 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_ca.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,393 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_eu.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,396 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_es.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,398 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_ca.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,400 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_en.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,403 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_en.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,405 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_ca.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,407 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_en.yaml -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:33,410 copying lm_eval/tasks/aime/aime24.yaml -> build/lib/lm_eval/tasks/aime 2026-05-11T15:52:33,412 copying lm_eval/tasks/aime/aime.yaml -> build/lib/lm_eval/tasks/aime 2026-05-11T15:52:33,414 copying lm_eval/tasks/aime/aime25.yaml -> build/lib/lm_eval/tasks/aime 2026-05-11T15:52:33,416 creating build/lib/lm_eval/tasks/eus_proficiency 2026-05-11T15:52:33,417 copying lm_eval/tasks/eus_proficiency/eus_proficiency.yaml -> build/lib/lm_eval/tasks/eus_proficiency 2026-05-11T15:52:33,420 copying lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,423 copying lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,425 copying lm_eval/tasks/mmlusr/question_only/question_only_human_aging.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,428 copying lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,430 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,433 copying lm_eval/tasks/mmlusr/question_only/question_only_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,435 copying lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,437 copying lm_eval/tasks/mmlusr/question_only/question_only_philosophy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,440 copying lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,442 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,445 copying lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,447 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,450 copying lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,452 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,455 copying lm_eval/tasks/mmlusr/question_only/question_only_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,457 copying lm_eval/tasks/mmlusr/question_only/question_only_international_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,460 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,462 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,465 copying lm_eval/tasks/mmlusr/question_only/question_only_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,467 copying lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,470 copying lm_eval/tasks/mmlusr/question_only/question_only_world_religions.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,472 copying lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,474 copying lm_eval/tasks/mmlusr/question_only/question_only_public_relations.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,477 copying lm_eval/tasks/mmlusr/question_only/question_only_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,480 copying lm_eval/tasks/mmlusr/question_only/question_only_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,482 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,484 copying lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,487 copying lm_eval/tasks/mmlusr/question_only/question_only_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,490 copying lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,492 copying lm_eval/tasks/mmlusr/question_only/question_only_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,495 copying lm_eval/tasks/mmlusr/question_only/question_only_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,498 copying lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,500 copying lm_eval/tasks/mmlusr/question_only/question_only_nutrition.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,503 copying lm_eval/tasks/mmlusr/question_only/question_only_security_studies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,505 copying lm_eval/tasks/mmlusr/question_only/question_only_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,507 copying lm_eval/tasks/mmlusr/question_only/question_only_prehistory.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,510 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,513 copying lm_eval/tasks/mmlusr/question_only/question_only_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,515 copying lm_eval/tasks/mmlusr/question_only/question_only_global_facts.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,518 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,521 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,523 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,525 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,528 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,530 copying lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,532 copying lm_eval/tasks/mmlusr/question_only/question_only_sociology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,535 copying lm_eval/tasks/mmlusr/question_only/question_only_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,537 copying lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,540 copying lm_eval/tasks/mmlusr/question_only/question_only_virology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,542 copying lm_eval/tasks/mmlusr/question_only/_question_only.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,545 copying lm_eval/tasks/mmlusr/question_only/question_only_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,547 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,550 copying lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,552 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,555 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,558 copying lm_eval/tasks/mmlusr/question_only/question_only_management.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,560 copying lm_eval/tasks/mmlusr/question_only/question_only_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,563 copying lm_eval/tasks/mmlusr/question_only/question_only_marketing.yaml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:33,566 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,568 copying lm_eval/tasks/mmlusr/answer_only/answer_only_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,571 copying lm_eval/tasks/mmlusr/answer_only/answer_only_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,573 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,576 copying lm_eval/tasks/mmlusr/answer_only/answer_only_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,578 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,581 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,584 copying lm_eval/tasks/mmlusr/answer_only/answer_only_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,587 copying lm_eval/tasks/mmlusr/answer_only/answer_only_computer_security.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,590 copying lm_eval/tasks/mmlusr/answer_only/answer_only_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,592 copying lm_eval/tasks/mmlusr/answer_only/answer_only_global_facts.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,595 copying lm_eval/tasks/mmlusr/answer_only/answer_only_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,597 copying lm_eval/tasks/mmlusr/answer_only/answer_only_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,600 copying lm_eval/tasks/mmlusr/answer_only/answer_only_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,602 copying lm_eval/tasks/mmlusr/answer_only/answer_only_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,604 copying lm_eval/tasks/mmlusr/answer_only/answer_only_sociology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,607 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,610 copying lm_eval/tasks/mmlusr/answer_only/answer_only_anatomy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,612 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,615 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,617 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,620 copying lm_eval/tasks/mmlusr/answer_only/answer_only_human_aging.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,622 copying lm_eval/tasks/mmlusr/answer_only/answer_only_prehistory.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,625 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,628 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,631 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,633 copying lm_eval/tasks/mmlusr/answer_only/answer_only_security_studies.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,636 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,638 copying lm_eval/tasks/mmlusr/answer_only/answer_only_econometrics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,640 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,643 copying lm_eval/tasks/mmlusr/answer_only/answer_only_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,645 copying lm_eval/tasks/mmlusr/answer_only/answer_only_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,648 copying lm_eval/tasks/mmlusr/answer_only/_answer_only.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,650 copying lm_eval/tasks/mmlusr/answer_only/answer_only_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,653 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,655 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,658 copying lm_eval/tasks/mmlusr/answer_only/answer_only_college_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,660 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,663 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,666 copying lm_eval/tasks/mmlusr/answer_only/answer_only_management.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,669 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,672 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,675 copying lm_eval/tasks/mmlusr/answer_only/answer_only_philosophy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,677 copying lm_eval/tasks/mmlusr/answer_only/answer_only_nutrition.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,679 copying lm_eval/tasks/mmlusr/answer_only/answer_only_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,682 copying lm_eval/tasks/mmlusr/answer_only/answer_only_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,684 copying lm_eval/tasks/mmlusr/answer_only/answer_only_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,686 copying lm_eval/tasks/mmlusr/answer_only/answer_only_astronomy.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,689 copying lm_eval/tasks/mmlusr/answer_only/answer_only_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,691 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_law.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,694 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,696 copying lm_eval/tasks/mmlusr/answer_only/answer_only_marketing.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,698 copying lm_eval/tasks/mmlusr/answer_only/answer_only_world_religions.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,701 copying lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,703 copying lm_eval/tasks/mmlusr/answer_only/answer_only_virology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,706 copying lm_eval/tasks/mmlusr/answer_only/answer_only_international_law.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,709 copying lm_eval/tasks/mmlusr/answer_only/answer_only_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,712 copying lm_eval/tasks/mmlusr/answer_only/answer_only_public_relations.yaml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:33,714 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,717 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,720 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,722 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,725 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_security_studies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,727 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,730 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,732 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_sociology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,734 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,737 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,739 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_astronomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,741 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_aging.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,744 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,746 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,748 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_computer_security.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,751 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_marketing.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,753 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,755 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,758 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,760 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,762 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_global_facts.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,765 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_international_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,767 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,769 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,772 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,774 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_law.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,777 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,779 copying lm_eval/tasks/mmlusr/question_and_answer/_question_and_answer.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,781 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,784 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,786 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_prehistory.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,789 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,791 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_management.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,793 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,795 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,798 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,800 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,802 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,805 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,807 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,809 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,812 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,814 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,817 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,819 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,821 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_philosophy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,824 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,826 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_anatomy.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,828 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_nutrition.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,831 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,833 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,836 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_biology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,839 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,842 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,844 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_public_relations.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,846 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_econometrics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,849 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,851 copying lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:33,854 copying lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,856 copying lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,859 copying lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,861 copying lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,864 copying lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,866 copying lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,869 copying lm_eval/tasks/code_x_glue/code-text/_codexglue.yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:33,871 creating build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,872 copying lm_eval/tasks/haerae/haerae_sn.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,874 copying lm_eval/tasks/haerae/haerae_hi.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,877 copying lm_eval/tasks/haerae/haerae_lw.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,879 copying lm_eval/tasks/haerae/haerae_rw.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,882 copying lm_eval/tasks/haerae/_haerae.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,884 copying lm_eval/tasks/haerae/haerae_gk.yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:33,887 copying lm_eval/tasks/squadv2/squadv2.yaml -> build/lib/lm_eval/tasks/squadv2 2026-05-11T15:52:33,890 copying lm_eval/tasks/galician_bench/truthfulqa_gl_mc1.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,892 copying lm_eval/tasks/galician_bench/belebele_glg_Latn.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,894 copying lm_eval/tasks/galician_bench/mgsm_direct_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,897 copying lm_eval/tasks/galician_bench/parafrases_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,899 copying lm_eval/tasks/galician_bench/truthfulqa_gl_mc2.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,902 copying lm_eval/tasks/galician_bench/galician_bench.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,904 copying lm_eval/tasks/galician_bench/galcola.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,907 copying lm_eval/tasks/galician_bench/xstorycloze_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,909 copying lm_eval/tasks/galician_bench/summarization_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,912 copying lm_eval/tasks/galician_bench/openbookqa_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,914 copying lm_eval/tasks/galician_bench/xnli_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,917 copying lm_eval/tasks/galician_bench/truthfulqa_gl_gen.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,919 copying lm_eval/tasks/galician_bench/paws_gl.yaml -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:33,921 copying lm_eval/tasks/galician_bench/flores_gl/flores_fr-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,924 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-eu.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,926 copying lm_eval/tasks/galician_bench/flores_gl/flores_eu-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,928 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-es.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,931 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-pt.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,933 copying lm_eval/tasks/galician_bench/flores_gl/flores_it-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,935 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-de.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,938 copying lm_eval/tasks/galician_bench/flores_gl/flores_en-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,940 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,942 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-it.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,945 copying lm_eval/tasks/galician_bench/flores_gl/flores_pt-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,947 copying lm_eval/tasks/galician_bench/flores_gl/flores_es-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,950 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-en.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,953 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-fr.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,956 copying lm_eval/tasks/galician_bench/flores_gl/flores_de-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,960 copying lm_eval/tasks/galician_bench/flores_gl/flores_gl-ca.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,963 copying lm_eval/tasks/galician_bench/flores_gl/flores_ca-gl.yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:33,966 creating build/lib/lm_eval/tasks/piqa 2026-05-11T15:52:33,968 copying lm_eval/tasks/piqa/piqa.yaml -> build/lib/lm_eval/tasks/piqa 2026-05-11T15:52:33,972 copying lm_eval/tasks/eq_bench/default.yaml -> build/lib/lm_eval/tasks/eq_bench 2026-05-11T15:52:33,976 copying lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml -> build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:52:33,978 copying lm_eval/tasks/eq_bench/multilingual/eqbench_es.yaml -> build/lib/lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:52:33,981 copying lm_eval/tasks/portuguese_bench/assin_paraphrase.yaml -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T15:52:33,983 copying lm_eval/tasks/portuguese_bench/portuguese_bench.yaml -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T15:52:33,986 copying lm_eval/tasks/portuguese_bench/assin_entailment.yaml -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T15:52:33,988 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_de-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:33,991 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-de.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:33,993 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-es.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:33,995 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_en-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:33,998 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-ca.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,000 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-fr.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,002 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-it.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,005 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-en.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,007 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-eu.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,009 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_gl-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,011 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_eu-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,013 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_it-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,015 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_es-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,018 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_ca-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,020 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-gl.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,022 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_fr-pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,024 copying lm_eval/tasks/portuguese_bench/flores_pt/flores_pt.yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:34,027 copying lm_eval/tasks/basqueglue/wic.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:34,029 copying lm_eval/tasks/basqueglue/bhtc.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:34,031 copying lm_eval/tasks/basqueglue/qnli.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:34,034 copying lm_eval/tasks/basqueglue/vaxx.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:34,036 copying lm_eval/tasks/basqueglue/coref.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:34,038 copying lm_eval/tasks/basqueglue/bec.yaml -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:34,040 creating build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,041 copying lm_eval/tasks/inverse_scaling/inverse_scaling_repetitive_algebra.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,044 copying lm_eval/tasks/inverse_scaling/inverse_scaling_winobias_antistereotype.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,046 copying lm_eval/tasks/inverse_scaling/inverse_scaling_into_the_unknown.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,048 copying lm_eval/tasks/inverse_scaling/inverse_scaling_memo_trap.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,050 copying lm_eval/tasks/inverse_scaling/inverse_scaling_quote_repetition.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,053 copying lm_eval/tasks/inverse_scaling/inverse_scaling_sig_figs.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,055 copying lm_eval/tasks/inverse_scaling/inverse_scaling_hindsight_neglect.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,057 copying lm_eval/tasks/inverse_scaling/inverse_scaling_modus_tollens.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,059 copying lm_eval/tasks/inverse_scaling/inverse_scaling_neqa.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,062 copying lm_eval/tasks/inverse_scaling/inverse_scaling_pattern_matching_suppression.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,064 copying lm_eval/tasks/inverse_scaling/inverse_scaling_redefine_math.yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:34,066 creating build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T15:52:34,067 copying lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_p_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T15:52:34,069 copying lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_q_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T15:52:34,071 copying lm_eval/tasks/kbl/reasoning/kbl_statement_consistency_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T15:52:34,073 copying lm_eval/tasks/kbl/reasoning/kbl_causal_reasoning_em.yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T15:52:34,075 creating build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,076 copying lm_eval/tasks/kbl/knowledge/kbl_statute_number_and_content_matching_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,079 copying lm_eval/tasks/kbl/knowledge/kbl_query_statute_matching_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,081 copying lm_eval/tasks/kbl/knowledge/kbl_offense_component_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,083 copying lm_eval/tasks/kbl/knowledge/kbl_statute_hallucination_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,085 copying lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,087 copying lm_eval/tasks/kbl/knowledge/kbl_legal_concept_qa_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,089 copying lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_reasoning_em.yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:34,091 creating build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,093 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,095 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2011.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,098 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,100 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,102 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2010.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,104 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,106 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,109 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,111 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,113 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,115 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,118 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,120 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,122 copying lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:34,124 creating build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,125 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2024.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,127 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,130 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,132 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,134 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,136 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,138 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,140 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,142 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,144 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,146 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,149 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,151 copying lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:34,153 creating build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,154 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,156 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,158 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2025.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,161 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,163 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,165 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,167 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,170 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,172 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,175 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,177 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2024.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,179 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,182 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,184 copying lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:34,186 creating build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,188 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2022.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,190 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2015.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,193 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2012.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,195 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2013.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,197 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2017.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,200 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2021.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,202 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2019.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,204 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2020.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,207 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2024.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,209 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2016.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,212 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2018.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,214 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2023.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,217 copying lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2014.yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:34,219 copying lm_eval/tasks/egyhellaswag/egyhellaswag.yaml -> build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T15:52:34,222 copying lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/lib/lm_eval/tasks/wmt2016 2026-05-11T15:52:34,224 copying lm_eval/tasks/mathqa/mathqa.yaml -> build/lib/lm_eval/tasks/mathqa 2026-05-11T15:52:34,227 copying lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T15:52:34,229 copying lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T15:52:34,232 copying lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T15:52:34,234 copying lm_eval/tasks/pisa/pisa_fr.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,237 copying lm_eval/tasks/pisa/pisa_ch.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,239 copying lm_eval/tasks/pisa/pisa_es_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,242 copying lm_eval/tasks/pisa/pisa_en.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,244 copying lm_eval/tasks/pisa/_pisa_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,247 copying lm_eval/tasks/pisa/pisa_en_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,249 copying lm_eval/tasks/pisa/pisa_fr_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,251 copying lm_eval/tasks/pisa/_pisa.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,254 copying lm_eval/tasks/pisa/pisa_it_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,257 copying lm_eval/tasks/pisa/pisa_de_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,260 copying lm_eval/tasks/pisa/pisa_ch_llm_judged.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,262 copying lm_eval/tasks/pisa/pisa_es.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,264 copying lm_eval/tasks/pisa/pisa_de.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,266 copying lm_eval/tasks/pisa/pisa_it.yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:34,268 creating build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,270 copying lm_eval/tasks/xstorycloze/default_ru.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,272 copying lm_eval/tasks/xstorycloze/default_ar.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,274 copying lm_eval/tasks/xstorycloze/default_id.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,276 copying lm_eval/tasks/xstorycloze/_xstorycloze.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,279 copying lm_eval/tasks/xstorycloze/default_sw.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,281 copying lm_eval/tasks/xstorycloze/default_zh.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,283 copying lm_eval/tasks/xstorycloze/default_my.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,286 copying lm_eval/tasks/xstorycloze/default_eu.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,288 copying lm_eval/tasks/xstorycloze/default_hi.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,290 copying lm_eval/tasks/xstorycloze/default_es.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,293 copying lm_eval/tasks/xstorycloze/default_te.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,295 copying lm_eval/tasks/xstorycloze/default_en.yaml -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:34,298 copying lm_eval/tasks/esbbq/esbbq_lgbtqia.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,300 copying lm_eval/tasks/esbbq/esbbq_age.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,302 copying lm_eval/tasks/esbbq/esbbq_nationality.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,305 copying lm_eval/tasks/esbbq/esbbq_ses.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,307 copying lm_eval/tasks/esbbq/esbbq_spanish_region.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,310 copying lm_eval/tasks/esbbq/esbbq_disability_status.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,312 copying lm_eval/tasks/esbbq/esbbq.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,314 copying lm_eval/tasks/esbbq/esbbq_religion.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,317 copying lm_eval/tasks/esbbq/esbbq_gender.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,319 copying lm_eval/tasks/esbbq/esbbq_race_ethnicity.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,322 copying lm_eval/tasks/esbbq/esbbq_physical_appearance.yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:34,324 creating build/lib/lm_eval/tasks/benchmarks 2026-05-11T15:52:34,325 copying lm_eval/tasks/benchmarks/openllm.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T15:52:34,327 copying lm_eval/tasks/benchmarks/t0_eval.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T15:52:34,330 copying lm_eval/tasks/benchmarks/minerva_math.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T15:52:34,332 copying lm_eval/tasks/benchmarks/pythia.yaml -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T15:52:34,334 creating build/lib/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T15:52:34,336 copying lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/lib/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T15:52:34,338 creating build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T15:52:34,339 copying lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T15:52:34,343 copying lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T15:52:34,346 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10proc.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,348 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,350 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,353 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9cm.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,355 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_atc.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,358 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,360 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,362 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9proc.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,365 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,367 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,370 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,372 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,374 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,377 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,379 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,381 copying lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10cm.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,383 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,386 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_hard.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,388 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,391 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_easy.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,394 copying lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_medium.yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:34,397 copying lm_eval/tasks/groundcocoa/groundcocoa.yaml -> build/lib/lm_eval/tasks/groundcocoa 2026-05-11T15:52:34,399 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,402 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,404 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,406 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,409 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,411 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,414 copying lm_eval/tasks/evalita_llm/_evalita-mp_ls_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,416 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,419 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,421 copying lm_eval/tasks/evalita_llm/_evalita-mp_re_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,424 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,426 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,429 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,431 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,433 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,436 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,438 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,441 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,443 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,446 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,448 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,450 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,453 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,455 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,458 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,461 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,464 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,467 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,469 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,472 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,474 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,476 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,478 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,481 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,483 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,486 copying lm_eval/tasks/evalita_llm/_evalita-mp.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,489 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,491 copying lm_eval/tasks/evalita_llm/_evalita-mp_ls_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,494 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,497 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,500 copying lm_eval/tasks/evalita_llm/_evalita-mp_re_task.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,502 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,505 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,507 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,509 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,511 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,514 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,516 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,518 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,521 copying lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,523 copying lm_eval/tasks/evalita_llm/_evalita-mp_re_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,525 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,528 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,530 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,533 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,535 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,538 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_p5.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,540 copying lm_eval/tasks/evalita_llm/_evalita-mp_mc.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,542 copying lm_eval/tasks/evalita_llm/_evalita-mp_wic_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,545 copying lm_eval/tasks/evalita_llm/_evalita-mp_faq_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,547 copying lm_eval/tasks/evalita_llm/_evalita-mp_ls_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,550 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,552 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p6.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,554 copying lm_eval/tasks/evalita_llm/_evalita-mp_te_tasks.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,557 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,559 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,562 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,564 copying lm_eval/tasks/evalita_llm/_evalita-mp_hs_p4.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,566 copying lm_eval/tasks/evalita_llm/_evalita-mp_sa_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,569 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p2.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,572 copying lm_eval/tasks/evalita_llm/_evalita-mp_gen.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,574 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p1.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,577 copying lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p3.yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:34,579 copying lm_eval/tasks/medtext/medtext.yaml -> build/lib/lm_eval/tasks/medtext 2026-05-11T15:52:34,582 copying lm_eval/tasks/medtext/medtext_perplexity.yaml -> build/lib/lm_eval/tasks/medtext 2026-05-11T15:52:34,584 creating build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:34,585 copying lm_eval/tasks/wmdp/wmdp_chem.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:34,587 copying lm_eval/tasks/wmdp/wmdp_bio.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:34,590 copying lm_eval/tasks/wmdp/wmdp_cyber.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:34,592 copying lm_eval/tasks/wmdp/_wmdp.yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:34,594 copying lm_eval/tasks/xquad/xquad_ro.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,597 copying lm_eval/tasks/xquad/xquad_el.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,599 copying lm_eval/tasks/xquad/xquad_de.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,601 copying lm_eval/tasks/xquad/xquad_ar.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,604 copying lm_eval/tasks/xquad/xquad_es.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,606 copying lm_eval/tasks/xquad/xquad_tr.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,609 copying lm_eval/tasks/xquad/xquad_vi.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,611 copying lm_eval/tasks/xquad/xquad_th.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,613 copying lm_eval/tasks/xquad/xquad_en.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,616 copying lm_eval/tasks/xquad/xquad_hi.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,618 copying lm_eval/tasks/xquad/xquad_ru.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,620 copying lm_eval/tasks/xquad/xquad_zh.yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:34,622 copying lm_eval/tasks/paws-x/paws_zh.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,625 copying lm_eval/tasks/paws-x/_pawsx.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,627 copying lm_eval/tasks/paws-x/paws_de.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,629 copying lm_eval/tasks/paws-x/paws_fr.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,632 copying lm_eval/tasks/paws-x/paws_es.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,634 copying lm_eval/tasks/paws-x/paws_en.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,636 copying lm_eval/tasks/paws-x/paws_ja.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,639 copying lm_eval/tasks/paws-x/paws_ko.yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:34,641 creating build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,642 copying lm_eval/tasks/pile/pile_github.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,644 copying lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,647 copying lm_eval/tasks/pile/pile_stackexchange.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,649 copying lm_eval/tasks/pile/pile_hackernews.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,652 copying lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,654 copying lm_eval/tasks/pile/pile_uspto.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,656 copying lm_eval/tasks/pile/pile_enron.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,659 copying lm_eval/tasks/pile/pile_books3.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,661 copying lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,664 copying lm_eval/tasks/pile/pile_wikipedia.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,666 copying lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,669 copying lm_eval/tasks/pile/pile_philpapers.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,671 copying lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,673 copying lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,676 copying lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,678 copying lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,680 copying lm_eval/tasks/pile/pile_pile-cc.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,683 copying lm_eval/tasks/pile/pile_europarl.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,685 copying lm_eval/tasks/pile/pile_arxiv.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,687 copying lm_eval/tasks/pile/pile_gutenberg.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,689 copying lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,691 copying lm_eval/tasks/pile/pile_freelaw.yaml -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:34,694 copying lm_eval/tasks/xcopa/default_it.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,696 copying lm_eval/tasks/xcopa/default_ta.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,698 copying lm_eval/tasks/xcopa/default_qu.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,700 copying lm_eval/tasks/xcopa/default_vi.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,702 copying lm_eval/tasks/xcopa/default_id.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,704 copying lm_eval/tasks/xcopa/_xcopa.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,706 copying lm_eval/tasks/xcopa/default_et.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,709 copying lm_eval/tasks/xcopa/default_th.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,711 copying lm_eval/tasks/xcopa/default_sw.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,713 copying lm_eval/tasks/xcopa/default_zh.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,715 copying lm_eval/tasks/xcopa/default_ht.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,717 copying lm_eval/tasks/xcopa/default_tr.yaml -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:34,719 creating build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,720 copying lm_eval/tasks/turblimp/determiners.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,723 copying lm_eval/tasks/turblimp/anaphor_agreement.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,725 copying lm_eval/tasks/turblimp/argument_structure_transitive.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,727 copying lm_eval/tasks/turblimp/scrambling.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,729 copying lm_eval/tasks/turblimp/turblimp_group.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,731 copying lm_eval/tasks/turblimp/argument_structure_ditransitive.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,733 copying lm_eval/tasks/turblimp/irregular_forms.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,736 copying lm_eval/tasks/turblimp/relative_clauses.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,738 copying lm_eval/tasks/turblimp/suspended_affixation.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,740 copying lm_eval/tasks/turblimp/nominalization.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,742 copying lm_eval/tasks/turblimp/npi_licensing.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,745 copying lm_eval/tasks/turblimp/island_effects.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,747 copying lm_eval/tasks/turblimp/passives.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,749 copying lm_eval/tasks/turblimp/binding.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,751 copying lm_eval/tasks/turblimp/quantifiers.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,754 copying lm_eval/tasks/turblimp/subject_agreement.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,756 copying lm_eval/tasks/turblimp/ellipsis.yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:34,758 creating build/lib/lm_eval/tasks/glue/qqp 2026-05-11T15:52:34,759 copying lm_eval/tasks/glue/qqp/default.yaml -> build/lib/lm_eval/tasks/glue/qqp 2026-05-11T15:52:34,761 creating build/lib/lm_eval/tasks/glue/qnli 2026-05-11T15:52:34,762 copying lm_eval/tasks/glue/qnli/default.yaml -> build/lib/lm_eval/tasks/glue/qnli 2026-05-11T15:52:34,764 creating build/lib/lm_eval/tasks/glue/wnli 2026-05-11T15:52:34,765 copying lm_eval/tasks/glue/wnli/default.yaml -> build/lib/lm_eval/tasks/glue/wnli 2026-05-11T15:52:34,767 creating build/lib/lm_eval/tasks/glue/sst2 2026-05-11T15:52:34,768 copying lm_eval/tasks/glue/sst2/default.yaml -> build/lib/lm_eval/tasks/glue/sst2 2026-05-11T15:52:34,770 creating build/lib/lm_eval/tasks/glue/rte 2026-05-11T15:52:34,771 copying lm_eval/tasks/glue/rte/default.yaml -> build/lib/lm_eval/tasks/glue/rte 2026-05-11T15:52:34,773 creating build/lib/lm_eval/tasks/glue/mrpc 2026-05-11T15:52:34,774 copying lm_eval/tasks/glue/mrpc/default.yaml -> build/lib/lm_eval/tasks/glue/mrpc 2026-05-11T15:52:34,776 copying lm_eval/tasks/glue/mnli/mismatch.yaml -> build/lib/lm_eval/tasks/glue/mnli 2026-05-11T15:52:34,779 copying lm_eval/tasks/glue/mnli/default.yaml -> build/lib/lm_eval/tasks/glue/mnli 2026-05-11T15:52:34,781 creating build/lib/lm_eval/tasks/glue/cola 2026-05-11T15:52:34,782 copying lm_eval/tasks/glue/cola/default.yaml -> build/lib/lm_eval/tasks/glue/cola 2026-05-11T15:52:34,785 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:34,788 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:34,790 copying lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:34,793 copying lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:34,795 copying lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:34,797 copying lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:34,799 copying lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:34,802 copying lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:34,804 copying lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:34,807 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:34,809 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:34,811 copying lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:34,814 copying lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:34,816 copying lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:34,819 copying lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:34,822 copying lm_eval/tasks/afrobench/afrobench.yaml -> build/lib/lm_eval/tasks/afrobench 2026-05-11T15:52:34,825 copying lm_eval/tasks/afrobench/afrobench-lite.yaml -> build/lib/lm_eval/tasks/afrobench 2026-05-11T15:52:34,828 copying lm_eval/tasks/afrobench/naijarc/naijarc.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T15:52:34,830 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:52:34,832 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:52:34,834 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:52:34,837 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:52:34,839 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:52:34,840 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:52:34,843 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:52:34,845 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:52:34,847 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:52:34,849 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:52:34,851 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:52:34,853 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:52:34,856 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:52:34,857 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:52:34,859 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:52:34,862 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:52:34,864 creating build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:52:34,866 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:52:34,869 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_yor.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:52:34,872 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_hau.yaml -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:52:34,875 copying lm_eval/tasks/afrobench/sib/sib.yaml -> build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T15:52:34,878 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,880 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,882 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,885 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,887 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,890 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,892 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,894 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,897 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,899 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,902 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,904 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,906 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,909 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,911 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,914 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,916 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,918 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,921 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,923 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,925 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,928 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,930 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,933 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,935 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,937 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,940 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,942 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,945 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,947 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,950 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,952 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,954 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,957 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,959 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,962 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,964 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,967 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,970 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,972 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,975 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,977 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,979 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,982 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,984 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,987 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,989 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,991 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,994 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,996 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:34,999 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:35,001 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:35,003 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:35,006 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:35,008 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:35,011 copying lm_eval/tasks/afrobench/sib/prompt_2/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:35,013 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,015 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,018 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,020 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,023 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,025 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,027 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,030 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,032 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,035 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,037 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,040 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,042 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,045 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,047 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,049 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,052 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,054 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,056 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,059 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,061 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,063 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,066 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,068 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,070 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,073 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,075 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,078 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,080 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,082 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,085 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,087 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,090 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,092 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,094 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,097 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,099 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,102 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,104 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,106 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,109 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,111 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,114 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,116 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,119 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,121 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,124 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,126 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,129 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,131 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,133 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,136 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,138 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,141 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,143 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,145 copying lm_eval/tasks/afrobench/sib/prompt_1/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:35,148 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,150 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,153 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,155 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,157 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,160 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,162 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,165 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,167 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,170 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,172 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,175 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,177 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,180 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,182 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,185 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,187 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,189 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,192 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,194 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,197 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,200 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,202 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,205 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,207 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,209 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,212 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,214 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,216 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,219 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,221 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,223 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,226 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,229 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,231 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,233 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,235 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,238 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,240 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,243 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,245 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,248 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,250 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,252 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,255 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,258 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,260 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,263 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,265 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,268 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,270 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,273 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,276 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,278 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,281 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,283 copying lm_eval/tasks/afrobench/sib/prompt_5/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:35,285 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,288 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,290 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,292 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,295 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,297 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,300 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,302 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,304 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,306 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,309 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,311 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,314 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,316 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,318 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,321 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,323 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,326 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,328 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,331 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,333 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,335 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,338 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,340 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,343 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,345 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,348 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,350 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,353 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,355 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,358 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,360 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,363 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,365 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,368 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,370 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,373 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,375 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,378 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,380 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,383 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,385 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,388 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,391 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,393 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,396 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,398 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,401 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,403 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,406 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,408 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,411 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,413 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,416 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,418 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,421 copying lm_eval/tasks/afrobench/sib/prompt_3/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:35,423 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kmb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,426 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_bam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,429 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,431 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_xho.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,434 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kab.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,436 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_hau.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,439 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_lug.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,442 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_sna.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,444 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_por.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,447 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_fon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,449 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_nus.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,452 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_eng.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,454 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_dik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,457 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_som.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,459 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_luo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,462 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_afr.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,464 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tir.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,466 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_sag.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,469 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_amh.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,471 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_run.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,473 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kik.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,475 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tzm.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,478 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_umb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,480 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_swa.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,483 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_aeb.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,485 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_arz.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,487 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,490 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_fra.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,492 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_lin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,495 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_nso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,497 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_taq.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,500 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,502 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kbp.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,505 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_wol.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,507 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_mos.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,510 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kea.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,512 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_aka.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,514 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_bem.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,517 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ary.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,519 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_plt.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,522 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,524 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_nya.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,526 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kam.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,529 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_sot.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,531 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tum.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,534 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_twi.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,536 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_knc.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,538 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_yor.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,541 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kon.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,543 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_zul.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,546 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_lua.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,548 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,551 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_cjk.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,553 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_dyu.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,556 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_tso.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,558 copying lm_eval/tasks/afrobench/sib/prompt_4/sib_kin.yaml -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:35,561 copying lm_eval/tasks/afrobench/masakhanews/masakhanews.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:52:35,563 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,566 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,568 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,571 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,573 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,575 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,578 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,580 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,583 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,585 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,587 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,590 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,592 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,595 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,597 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,600 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:35,602 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,606 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,609 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,611 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,615 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,617 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,620 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,622 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,624 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,627 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,629 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,632 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,634 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,637 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,639 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,641 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:35,644 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,647 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,649 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,651 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,654 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,656 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,659 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,661 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,664 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,666 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,668 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,671 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,673 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,676 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,678 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,680 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:35,683 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,685 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,688 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,690 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,693 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,695 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,697 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,700 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,702 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,705 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,707 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,710 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,712 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,714 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,716 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,719 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:35,721 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,724 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,726 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_run.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,728 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,731 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,733 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_orm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,735 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_amh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,738 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,740 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_tir.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,743 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,745 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,748 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_fra.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,750 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_eng.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,753 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,756 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,758 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_som.yaml -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:35,761 copying lm_eval/tasks/afrobench/belebele/belebele.yaml -> build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T15:52:35,764 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,765 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,767 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,770 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,772 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,775 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,777 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,780 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,782 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,785 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,787 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,789 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,792 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,794 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,796 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,799 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,801 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,804 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,806 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,808 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,811 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,813 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,816 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,818 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,820 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,823 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,825 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,828 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,830 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,832 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,835 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,837 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:35,840 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,841 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,843 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,846 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,848 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,850 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,853 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,855 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,857 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,860 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,862 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,864 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,867 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,869 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,872 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,874 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,876 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,879 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,881 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,884 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,886 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,888 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,891 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,893 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,896 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,898 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,900 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,903 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,905 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,908 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,910 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,912 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:35,914 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,915 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,919 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,921 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,924 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,926 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,929 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,931 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,934 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,936 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,939 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,941 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,944 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,946 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,949 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,952 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,954 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,957 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,960 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,962 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,964 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,967 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,970 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,972 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,974 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,977 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,979 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,982 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,984 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,987 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,990 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,992 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:35,994 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:35,995 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:35,998 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,000 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,003 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,005 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,007 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,010 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,012 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,014 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,017 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,020 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,022 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,025 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,027 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,030 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,033 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,035 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,038 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,040 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,042 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,045 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,047 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,050 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,052 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,054 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,056 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,059 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,061 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,064 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,067 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,069 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:36,072 creating build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,073 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kea.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,077 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_zul.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,079 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sot.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,082 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tso.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,084 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,086 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ary.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,089 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_swa.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,092 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_som.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,094 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_arz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,096 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_afr.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,099 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_hau.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,101 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_wol.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,104 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_nya.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,106 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_plt.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,108 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ssw.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,110 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_gaz.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,113 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,115 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_eng.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,118 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tir.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,120 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_amh.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,123 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_xho.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,126 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_bam.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,128 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fuv.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,131 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fra.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,133 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,135 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lug.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,138 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sna.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,140 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_luo.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,143 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lin.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,145 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_por.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,147 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele_yor.yaml -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:36,150 copying lm_eval/tasks/afrobench/masakhapos/masakhapos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:52:36,152 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,155 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,157 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,160 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,162 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,164 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,166 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,169 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,171 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,174 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,176 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,178 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,181 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,183 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,185 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,188 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,190 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,192 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,195 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,197 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:36,200 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,202 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,204 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,207 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,209 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,212 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,214 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,217 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,219 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,222 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,224 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,227 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,229 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,232 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,234 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,236 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,239 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,241 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,244 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,246 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:36,248 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,251 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,253 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,255 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,258 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,260 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,263 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,265 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,268 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,270 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,273 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,275 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,278 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,280 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,282 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,285 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,287 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,290 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,293 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,295 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:36,298 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,300 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,303 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,305 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,307 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,310 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,312 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,315 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,317 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,319 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,322 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,324 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,326 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,329 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,331 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,334 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,336 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,338 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,341 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,343 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:36,346 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_wol.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,348 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_nya.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,351 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_sna.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,353 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,356 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,358 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,361 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,363 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_lug.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,366 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_swa.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,368 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_hau.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,370 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yor.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,373 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,375 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_kin.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,377 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_xho.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,380 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_fon.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,382 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_tsn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,385 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bam.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,387 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_twi.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,390 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,392 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_zul.yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:36,395 copying lm_eval/tasks/afrobench/salt/salt.yaml -> build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T15:52:36,397 creating build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,398 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lug.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,401 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-teo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,403 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,405 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_teo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,408 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_lug-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,410 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lgg.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,413 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-nyn.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,415 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-swa.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,418 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ach.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,420 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_swa-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,423 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_nyn-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,425 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_lgg-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,428 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_ach-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,430 copying lm_eval/tasks/afrobench/salt/prompt_2/salt_ibo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:36,432 creating build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,433 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lug.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,436 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-teo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,438 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,441 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_teo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,443 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_lug-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,445 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lgg.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,448 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-nyn.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,450 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-swa.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,453 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ach.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,455 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_swa-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,457 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_nyn-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,460 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_lgg-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,462 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_ach-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,464 copying lm_eval/tasks/afrobench/salt/prompt_1/salt_ibo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:36,466 creating build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,468 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lug.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,470 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-teo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,473 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,475 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_teo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,478 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_lug-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,480 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lgg.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,483 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-nyn.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,485 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-swa.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,488 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ach.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,490 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_swa-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,493 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_nyn-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,495 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_lgg-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,498 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_ach-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,500 copying lm_eval/tasks/afrobench/salt/prompt_3/salt_ibo-eng.yaml -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:36,503 copying lm_eval/tasks/afrobench/ntrex/ntrex.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T15:52:36,505 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,507 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,510 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,512 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,515 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,517 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,519 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,522 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,524 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,527 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,529 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,531 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,533 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,536 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,538 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,541 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,543 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,545 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,547 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,550 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,552 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,555 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,557 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,559 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,561 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,563 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,566 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,568 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,570 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,572 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,574 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,576 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,579 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:36,581 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,582 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,585 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,587 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,589 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,592 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,594 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,597 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,599 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,601 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,604 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,606 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,609 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,611 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,614 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,616 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,619 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,622 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,625 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,627 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,631 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,633 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,636 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,639 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,641 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,644 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,646 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,649 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,651 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,654 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,657 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,659 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,662 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:36,664 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,666 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,668 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,671 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,673 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,676 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,679 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,682 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,685 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,687 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,690 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,693 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,696 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,699 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,702 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,705 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,708 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,710 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,713 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,716 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,718 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,721 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,723 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,726 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,728 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,730 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,732 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,735 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,737 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,739 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,741 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,743 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,746 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:36,748 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,749 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,752 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,754 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,757 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,759 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,762 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,764 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,766 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,769 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,771 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,774 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,777 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,779 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,782 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,784 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,786 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,789 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,791 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,795 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,798 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,800 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,803 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,805 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,808 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,810 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,813 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,815 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,818 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,820 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,823 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,825 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,828 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:36,830 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,832 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,834 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,836 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,839 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,841 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,843 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,846 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,848 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,850 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,852 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,855 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,857 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,860 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,862 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,865 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,867 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,870 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,872 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,874 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,877 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,879 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,881 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,884 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,886 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,888 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,891 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,893 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,895 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,898 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,900 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,902 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,905 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:36,907 creating build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,908 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,911 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,913 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,915 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,918 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,920 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,922 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,925 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,927 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,930 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,932 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,935 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,937 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,940 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,942 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,945 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,947 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,949 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,952 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,954 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,956 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,959 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,961 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,964 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,966 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,969 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,971 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,974 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,976 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,979 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,981 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,984 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:36,986 copying lm_eval/tasks/afrobench/flores/flores.yaml -> build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T15:52:36,989 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:36,990 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_luo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:36,993 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:36,996 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:36,998 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,000 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,003 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,005 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,008 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lug_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,010 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,013 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lua_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,015 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_run_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,017 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kea_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,020 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,022 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,025 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kab_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,027 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,029 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,032 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,034 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aka_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,037 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ary_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,039 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,042 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_plt_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,044 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,046 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,049 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_mos_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,051 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sun_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,053 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,056 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_swh_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,058 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,061 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,063 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_umb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,065 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,068 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,070 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,073 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ban_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,075 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,078 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,080 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,083 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sot_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,085 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,087 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,090 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,092 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,095 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nus_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,097 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,100 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,102 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,105 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,107 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,110 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tum_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,112 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_acq_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,114 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,117 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_twi_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,119 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,122 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_arz_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,124 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sag_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,127 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,129 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,131 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,134 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,136 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,139 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:37,141 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,142 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,145 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lua_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,147 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,150 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,152 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aka_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,155 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,157 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,160 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,162 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-umb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,164 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sot_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,167 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lug_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,169 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,171 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-run_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,174 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,176 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,179 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,182 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,184 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sag_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,187 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,190 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,193 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,195 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,198 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,200 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,203 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-luo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,205 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,207 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,210 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,212 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,215 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,217 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nus_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,220 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,222 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,225 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,228 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,231 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,233 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-acq_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,236 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,238 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,241 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,243 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,246 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,248 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-twi_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,251 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,253 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ary_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,256 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,258 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,261 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ban_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,264 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kea_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,266 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kab_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,269 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,272 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,275 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-plt_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,277 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,280 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tum_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,282 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-mos_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,284 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,287 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,289 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,291 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-swh_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,294 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sun_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,296 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-arz_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:37,298 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,300 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_luo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,303 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,305 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,307 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,310 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,313 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,315 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,318 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lug_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,320 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,322 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lua_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,324 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_run_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,327 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kea_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,329 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,332 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,334 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kab_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,336 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,339 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,341 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,343 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aka_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,346 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ary_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,348 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,351 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_plt_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,353 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,355 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,358 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_mos_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,360 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sun_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,362 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,365 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_swh_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,367 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,370 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,372 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_umb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,374 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,376 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,379 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,381 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ban_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,383 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,385 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,387 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,390 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sot_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,392 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,394 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,397 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,399 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,402 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nus_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,404 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,406 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,409 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,411 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,414 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,416 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tum_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,419 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_acq_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,421 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,423 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_twi_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,425 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,428 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_arz_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,430 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sag_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,432 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,435 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,437 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,440 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,442 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,444 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:37,447 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,448 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,450 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lua_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,453 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,455 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,457 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aka_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,460 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,462 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,465 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,467 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-umb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,470 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sot_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,472 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lug_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,475 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,477 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-run_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,480 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,482 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,484 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,487 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,489 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sag_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,492 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,494 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,497 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,499 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,501 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,504 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,506 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-luo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,509 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,511 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,513 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,516 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,518 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,520 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nus_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,523 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,525 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,527 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,529 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,532 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,534 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-acq_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,536 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,538 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,541 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,543 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,545 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,547 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-twi_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,550 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,552 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ary_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,554 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,557 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,559 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ban_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,561 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kea_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,564 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kab_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,566 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,569 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,571 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-plt_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,573 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,576 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tum_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,578 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-mos_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,580 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,582 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,584 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,587 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-swh_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,589 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sun_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,591 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-arz_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:37,593 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,595 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_luo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,597 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,600 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_yor_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,602 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,604 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_zul_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,607 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,609 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,611 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lug_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,614 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_som_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,616 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lua_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,619 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_run_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,621 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kea_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,624 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,626 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,628 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kab_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,631 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,633 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,635 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,637 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aka_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,639 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ary_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,641 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,644 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_plt_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,646 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_hau_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,648 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,650 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_mos_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,653 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sun_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,655 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,658 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_swh_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,660 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,663 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,665 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_umb_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,667 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bem_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,670 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,673 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,675 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ban_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,677 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,680 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,682 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,685 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sot_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,687 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nya_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,689 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_xho_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,692 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sna_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,694 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,697 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nus_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,699 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tso_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,702 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kam_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,704 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lin_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,707 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,709 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_wol_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,712 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tum_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,714 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_acq_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,717 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,719 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_twi_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,721 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,723 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_arz_Arab-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,726 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sag_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,728 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,731 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dik_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,733 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kon_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,735 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fra_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,737 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_afr_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,740 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Latn-eng_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:37,742 creating build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,743 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,746 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lua_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,748 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,750 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,752 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aka_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,755 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,757 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-hau_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,760 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-xho_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,762 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-umb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,764 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sot_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,767 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lug_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,769 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,771 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-run_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,774 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fra_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,776 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,778 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,781 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,783 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sag_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,786 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,788 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,790 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sna_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,793 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,795 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,797 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-som_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,800 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-luo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,802 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tso_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,805 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-wol_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,807 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,809 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,812 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-zul_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,814 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nus_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,817 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bem_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,819 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kin_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,821 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,824 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,826 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,829 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-acq_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,831 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,833 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kik_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,836 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,838 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,840 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,843 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-twi_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,845 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,847 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ary_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,850 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nya_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,852 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,855 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ban_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,857 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kea_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,859 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kab_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,862 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-afr_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,864 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,866 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-plt_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,869 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bam_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,871 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tum_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,874 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-mos_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,876 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-yor_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,879 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,881 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kon_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,884 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-swh_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,887 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sun_Latn.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,891 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-arz_Arab.yaml -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:37,894 copying lm_eval/tasks/afrobench/injongointent/injongointent.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T15:52:37,896 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,898 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,901 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,903 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,906 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,908 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,910 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,913 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,915 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,917 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,920 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,922 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,925 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,927 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,930 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,932 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,934 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:37,937 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,939 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,942 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,944 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,946 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,949 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,951 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,953 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,956 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,958 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,960 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,963 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,965 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,968 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,970 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,972 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,975 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:37,977 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,980 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,982 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,985 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,987 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,989 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,992 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,994 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,997 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:37,999 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,002 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,004 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,006 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,008 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,011 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,013 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,015 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:38,018 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,020 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,022 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,025 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,027 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,029 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,032 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,034 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,037 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,039 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,042 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,044 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,047 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,049 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,052 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,054 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,057 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:38,060 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,062 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sna.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,064 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,067 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ewe.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,069 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_wol.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,071 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_amh.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,074 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_orm.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,076 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_kin.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,078 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_eng.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,081 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_yor.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,083 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_swa.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,085 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_hau.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,088 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sot.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,091 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_zul.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,093 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_xho.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,095 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_twi.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,098 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lug.yaml -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:38,101 copying lm_eval/tasks/afrobench/nollysenti/nollysenti.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti 2026-05-11T15:52:38,103 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:38,106 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:38,108 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:38,110 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:38,113 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:38,115 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:38,118 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:38,120 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:38,122 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:38,125 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:38,127 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:38,130 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:38,132 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:38,134 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:38,136 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:38,139 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:38,142 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:38,144 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:38,146 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:38,149 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:38,151 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:38,154 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:38,156 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_eng.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:38,159 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:38,162 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:38,164 copying lm_eval/tasks/afrobench/afrisenti/afrisenti.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:52:38,166 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,169 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,172 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,174 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,177 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,179 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,182 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,185 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,187 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,190 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,192 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,194 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,197 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,199 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:38,202 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,204 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,206 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,209 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,211 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,214 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,216 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,219 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,221 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,223 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,226 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,229 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,232 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,234 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:38,237 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,239 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,242 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,244 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,246 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,249 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,251 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,254 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,257 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,259 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,261 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,264 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,266 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,269 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:38,271 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,274 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,276 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,278 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,281 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,283 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,286 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,288 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,291 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,293 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,295 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,298 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,300 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,302 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:38,305 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_orm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,307 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,309 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ary.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,312 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,314 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_arq.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,316 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,319 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,321 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tir.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,324 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,326 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tso.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,329 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,331 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,333 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_por.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,335 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_amh.yaml -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:38,338 copying lm_eval/tasks/afrobench/adr/afridiacritics.yaml -> build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T15:52:38,340 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:38,341 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:38,344 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:38,346 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:38,349 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:38,351 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:38,353 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:38,355 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:38,357 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:38,360 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:38,362 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:38,365 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:38,367 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:38,368 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:38,371 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:38,373 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:38,375 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:38,378 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:38,380 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:38,382 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:38,384 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:38,387 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:38,390 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:38,392 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:38,394 creating build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:38,395 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_fon.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:38,397 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_wol.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:38,399 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yor.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:38,402 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:38,404 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:38,406 copying lm_eval/tasks/afrobench/xlsum/xlsum.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T15:52:38,409 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_somali.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,411 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_pidgin.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,414 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_igbo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,416 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_kirundi.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,418 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_hausa.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,421 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_tigrinya.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,423 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_oromo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,426 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_arabic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,429 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_amharic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,432 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_yoruba.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,434 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_swahili.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,436 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_telugu.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:38,439 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_somali.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,441 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_pidgin.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,443 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_igbo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,446 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_kirundi.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,448 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_hausa.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,450 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_tigrinya.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,453 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_oromo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,455 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_arabic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,458 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_amharic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,460 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_yoruba.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,462 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_swahili.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,465 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_telugu.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:38,467 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_somali.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,470 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_pidgin.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,473 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_igbo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,476 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_kirundi.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,478 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_hausa.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,480 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_tigrinya.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,482 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_oromo.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,485 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_arabic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,487 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_amharic.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,489 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_yoruba.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,491 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_swahili.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,494 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_telugu.yaml -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:38,496 copying lm_eval/tasks/afrobench/uhura-arc-easy/uhura.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:52:38,499 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,501 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,503 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,505 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,508 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,510 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,512 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:38,514 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,517 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,519 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,522 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,524 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,526 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,529 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:38,531 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,533 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,535 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,537 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,540 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,542 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,544 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:38,546 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,548 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,551 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,553 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,555 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,558 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,560 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:38,563 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_am.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,565 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_ha.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,567 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_yo.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,569 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_en.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,572 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_nso.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,574 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_sw.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,577 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_zu.yaml -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:38,579 copying lm_eval/tasks/afrobench/masakhaner/masakhaner.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:52:38,582 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,584 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,586 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,589 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,591 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,593 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,595 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,598 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,600 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,602 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,605 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,607 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,609 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,612 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,614 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,616 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,619 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,621 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,624 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,626 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:38,629 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,631 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,633 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,636 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,638 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,640 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,643 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,645 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,647 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,650 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,652 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,655 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,658 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,660 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,663 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,665 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,668 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,670 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,673 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,675 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:38,677 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,680 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,682 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,684 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,687 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,690 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,692 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,695 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,697 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,700 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,702 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,704 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,707 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,710 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,712 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,715 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,717 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,720 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,722 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,725 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:38,727 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,729 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,732 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,734 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,737 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,739 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,742 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,744 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,747 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,749 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,751 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,754 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,756 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,759 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,761 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,763 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,766 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,768 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,771 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,773 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:38,776 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,778 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_yo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,780 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,782 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,785 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_am.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,787 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_mos.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,789 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ha.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,792 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bbj.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,794 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_xh.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,796 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_pcm.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,799 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_wo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,801 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ee.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,804 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_lg.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,806 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_luo.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,808 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ny.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,811 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,813 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_rw.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,816 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ig.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,818 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tn.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,821 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_zu.yaml -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:38,823 copying lm_eval/tasks/afrobench/afriqa/afriqa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T15:52:38,826 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,828 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,831 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,833 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,836 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,838 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,840 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,842 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,845 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:38,847 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,849 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,852 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,854 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,856 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,858 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,861 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,863 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,866 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:38,868 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,870 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,873 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,875 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,877 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,880 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,882 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,885 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,887 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:38,890 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,892 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,895 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,897 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,900 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,903 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,905 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,907 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,910 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:38,912 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_zul.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,915 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_twi.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,920 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_kin.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,922 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_hau.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,924 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_bem.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,927 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_swa.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,929 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_ibo.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,931 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_yor.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,934 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_fon.yaml -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:38,936 copying lm_eval/tasks/afrobench/mafand/mafand.yaml -> build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T15:52:38,938 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_kin-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,941 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_pcm-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,943 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_mos-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,946 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bam-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,948 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_sna-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,951 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_lug-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,953 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_fon-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,955 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ibo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,958 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bbj-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,960 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_yor-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,963 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_xho-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,965 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ewe-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,968 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_swa-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,970 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_amh-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,973 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_luo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,975 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_tsn-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,977 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_nya-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,979 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_zul-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,982 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_twi-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,984 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_wol-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,986 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_hau-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:38,989 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-twi.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:38,991 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-luo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:38,994 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bam.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:38,996 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-nya.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:38,999 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-sna.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,001 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-pcm.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,003 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-fon.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,006 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,008 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-ewe.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,011 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-yor.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,013 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-swa.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,016 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-kin.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,019 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-hau.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,021 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-lug.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,024 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-mos.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,027 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-tsn.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,029 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-xho.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,031 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-zul.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,034 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-wol.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,036 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bbj.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,038 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-amh.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:39,041 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_kin-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,043 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_pcm-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,046 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_mos-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,049 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bam-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,051 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_sna-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,053 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_lug-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,056 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_fon-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,058 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ibo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,061 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bbj-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,063 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_yor-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,065 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_xho-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,068 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ewe-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,070 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_swa-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,073 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_amh-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,075 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_luo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,077 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_tsn-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,080 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_nya-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,082 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_zul-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,084 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_twi-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,087 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_wol-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,089 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_hau-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:39,092 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-twi.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,094 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-luo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,096 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bam.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,099 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-nya.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,101 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-sna.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,104 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-pcm.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,106 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-fon.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,108 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,111 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-ewe.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,113 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-yor.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,116 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-swa.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,118 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-kin.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,120 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-hau.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,122 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-lug.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,125 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-mos.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,127 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-tsn.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,130 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-xho.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,132 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-zul.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,135 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-wol.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,137 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bbj.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,139 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-amh.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:39,142 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_kin-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,144 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_pcm-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,147 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_mos-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,149 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bam-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,151 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_sna-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,154 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_lug-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,156 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_fon-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,159 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ibo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,161 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bbj-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,163 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_yor-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,166 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_xho-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,168 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ewe-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,170 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_swa-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,173 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_amh-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,175 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_luo-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,178 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_tsn-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,180 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_nya-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,182 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_zul-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,185 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_twi-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,187 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_wol-fr.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,190 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_hau-en.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:39,192 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-twi.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,195 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-luo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,197 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bam.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,200 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-nya.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,202 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-sna.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,204 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-pcm.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,207 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-fon.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,209 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-ibo.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,211 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-ewe.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,214 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-yor.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,217 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-swa.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,219 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-kin.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,221 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-hau.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,224 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-lug.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,226 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-mos.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,229 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-tsn.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,231 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-xho.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,234 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-zul.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,236 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-wol.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,238 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bbj.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,241 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-amh.yaml -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:39,243 copying lm_eval/tasks/afrobench/openai_mmlu/openai_mmlu.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:52:39,246 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:52:39,247 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:52:39,249 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:52:39,252 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:52:39,254 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:52:39,255 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:52:39,257 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:52:39,260 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:52:39,262 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:52:39,263 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:52:39,266 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:52:39,268 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:52:39,270 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:52:39,271 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:52:39,274 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:52:39,276 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:52:39,278 creating build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:52:39,280 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_swa.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:52:39,282 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_ara.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:52:39,285 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_yor.yaml -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:52:39,287 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,290 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,292 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,295 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,297 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,299 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,302 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,304 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,307 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,309 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,312 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,314 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,316 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,318 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,321 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,323 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,325 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,328 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,330 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,333 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,335 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,338 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,340 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,343 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,345 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,348 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,350 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,353 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,355 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,357 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,360 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,362 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,365 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,367 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,370 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,372 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,375 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,377 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,379 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,382 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,384 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,386 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,388 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,391 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,393 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,395 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,397 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,400 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,402 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,405 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,407 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,409 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,412 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,414 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,416 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,419 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,422 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,424 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,427 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,429 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,431 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,434 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,436 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,439 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,442 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,444 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,446 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,449 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,451 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,453 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,456 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,458 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,460 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,463 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,465 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,468 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,470 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,472 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,475 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,477 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,480 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,482 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,484 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,487 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,489 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,492 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,494 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,497 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,499 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,501 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,504 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,506 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,509 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,511 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,513 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,516 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,518 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,520 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,523 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,525 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,527 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,529 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,532 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,534 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,537 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,539 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,541 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,544 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,546 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,548 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,551 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,553 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,556 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,558 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,560 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,563 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,565 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,567 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,570 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,572 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,575 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,577 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,579 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,582 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,585 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,587 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,590 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,592 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,595 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,597 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,600 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,602 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,604 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,607 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,609 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,612 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,614 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,616 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,619 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,622 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,624 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,627 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,630 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,633 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,636 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,638 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,640 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,642 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,645 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,647 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,649 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,652 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,654 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,656 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,659 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,661 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,664 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,667 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,670 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,673 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,676 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,678 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,681 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,683 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,685 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,688 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,690 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,693 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,695 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,698 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,700 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,703 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,706 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,708 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,710 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,713 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,715 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,717 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,720 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,722 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,724 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,726 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,729 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,731 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,733 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,736 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,738 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,740 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,743 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,745 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,748 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,750 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,753 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,755 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,757 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,760 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,762 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,764 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,767 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,769 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,772 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,774 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,776 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,779 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,782 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,784 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,786 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,789 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,791 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,793 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,796 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,798 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,800 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,803 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,805 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,808 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,810 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,813 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,815 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,818 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,820 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,822 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,825 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,827 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,830 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,832 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,835 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,837 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,839 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,841 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,844 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,846 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,848 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,851 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,853 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,855 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,858 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,860 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,863 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,865 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,868 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,870 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,873 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,875 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,877 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,880 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,882 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,884 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,887 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,890 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,892 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,895 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,897 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,899 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,902 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,904 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,906 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,908 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,911 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,913 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,915 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,917 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,920 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,922 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,925 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,927 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,929 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,932 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,934 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,936 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,939 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,941 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,944 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,946 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,948 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,951 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,953 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,956 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,958 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,961 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,963 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,965 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,967 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,970 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,972 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,974 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,977 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,979 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,982 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,984 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,986 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,989 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,991 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,994 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,996 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:39,999 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,001 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,003 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,006 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,008 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,011 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,013 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,016 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,018 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,021 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,024 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,026 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,029 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,031 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,033 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,036 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,038 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,041 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,043 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,046 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,048 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,050 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,053 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,055 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,058 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,060 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,063 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,065 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,067 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,070 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,072 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,075 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,077 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,080 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,082 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,085 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,088 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,090 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,093 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,095 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,098 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,100 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,102 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,105 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,107 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,110 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,112 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,114 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,117 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,119 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,121 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,124 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,126 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,129 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,131 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,134 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,136 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,138 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,141 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,143 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,146 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,148 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,150 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,153 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,155 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,158 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,160 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,162 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,165 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,167 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,170 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,172 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,174 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,177 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,179 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,182 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,184 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,187 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,189 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,192 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,194 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,197 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,199 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,201 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,204 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,206 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,209 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,211 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,214 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,216 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,220 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,223 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,227 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,231 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,235 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,237 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,239 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,242 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,244 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,246 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,249 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,251 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,254 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,256 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,258 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,261 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,263 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,266 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,268 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,271 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,273 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,275 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,278 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,280 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,282 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,285 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,288 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,291 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,295 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,298 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,301 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,305 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,308 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,311 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,314 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,318 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,321 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,324 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,328 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,331 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,334 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,337 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,341 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,344 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,348 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,351 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,353 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,356 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,358 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,360 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,363 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,365 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,368 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,370 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,373 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,376 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,378 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,381 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,383 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,385 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,388 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,390 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,393 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,395 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,397 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,400 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,402 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,404 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,407 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,409 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,412 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,414 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,417 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,419 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,421 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,424 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,426 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,429 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,432 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,434 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,437 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,439 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,441 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,444 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,446 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,449 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,451 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,453 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,456 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,458 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,460 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,463 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,465 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,468 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,470 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,473 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,475 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,477 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,480 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,483 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,485 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,488 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,490 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,493 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,495 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,497 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,500 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,502 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,505 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,507 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,510 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,512 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,514 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,517 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,519 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,522 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,524 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,526 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,529 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,531 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,533 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,536 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,538 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,541 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,543 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,545 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,548 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,550 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,552 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,555 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,557 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,559 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,561 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,564 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,566 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,568 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,571 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,573 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,576 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,578 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,580 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,583 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,585 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,588 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,590 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,593 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,595 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,597 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,600 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,602 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,605 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,607 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,610 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,612 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,615 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,617 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,620 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,622 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,625 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,627 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,630 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,632 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,635 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,638 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,640 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,642 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,645 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,647 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,650 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,652 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,654 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,657 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,659 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,662 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,664 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,666 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,669 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,671 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,674 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,676 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,678 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,681 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,683 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,686 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,688 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,691 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,693 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,696 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,698 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,701 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,703 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,706 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,708 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,711 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,713 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,715 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,718 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,720 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,723 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,725 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,727 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,730 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,732 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,734 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,737 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,739 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,742 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,744 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,747 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,749 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,752 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,754 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,757 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,759 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,762 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,764 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,767 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,769 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,772 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,774 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,777 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,779 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,782 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,785 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,787 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,790 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,792 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,795 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,797 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,799 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,802 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,805 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,807 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,810 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,813 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,815 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,818 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,821 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,824 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,827 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,831 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,834 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,836 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,839 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,842 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,845 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,847 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,850 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,852 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,855 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,857 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,860 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,863 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,865 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,868 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,871 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,874 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,877 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,879 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,882 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,885 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,887 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,890 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,892 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,895 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,898 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,900 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,903 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,906 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,909 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,911 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,914 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,917 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,919 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,922 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,925 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,927 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,930 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,932 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,934 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,937 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,939 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,942 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,944 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,946 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,949 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,951 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,954 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_logical_fallacies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,956 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,959 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,962 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,964 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,967 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,970 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,972 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,974 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,977 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,980 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,982 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,985 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,987 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_philosophy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,989 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,992 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,994 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,996 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:40,999 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,001 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,004 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,006 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,009 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,011 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,013 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,016 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,018 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,020 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,023 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,025 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,027 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,030 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,033 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,035 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_formal_logic.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,038 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,040 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,043 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,045 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,048 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,050 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,053 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,055 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,058 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,060 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,063 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,065 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,068 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_conceptual_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,070 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,073 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,075 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,077 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,080 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,082 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,085 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,087 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,089 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,092 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,094 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,097 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,099 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_sexuality.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,102 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,104 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,107 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,110 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,112 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,115 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,117 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,120 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,122 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,125 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,127 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_humanities.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,129 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,132 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,134 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,136 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,139 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,141 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_international_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,143 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,146 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,148 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,151 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,153 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,156 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,158 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,160 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,163 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,165 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,168 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,171 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,174 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,176 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,179 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,181 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_security_studies.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,184 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,186 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,189 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,191 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,193 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,196 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,198 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,201 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,203 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,205 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_econometrics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,208 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,210 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,213 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_social_sciences.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,215 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,218 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,220 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_world_religions.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,223 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_statistics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,226 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,229 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_abstract_algebra.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,231 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,234 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,236 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,238 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,241 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,243 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,246 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_astronomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,248 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,251 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,253 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,256 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,258 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_geography.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,260 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,263 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_prehistory.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,265 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_stem.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,268 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,270 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,273 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,275 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,277 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,280 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,282 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,285 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,287 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_us_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,289 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,292 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_anatomy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,294 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,297 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,299 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,301 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,304 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,306 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_sociology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,309 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,311 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,313 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,316 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,318 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,321 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,323 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_miscellaneous.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,326 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,328 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,330 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,333 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,335 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_disputes.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,338 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,340 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,342 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_nutrition.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,345 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,347 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,350 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_electrical_engineering.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,352 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,355 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,357 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_world_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,360 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_law.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,362 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_medical_genetics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,364 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,367 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,369 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,372 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_jurisprudence.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,374 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,376 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_psychology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,379 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,381 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_biology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,383 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,386 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,388 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_marketing.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,391 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,393 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_aging.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,396 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_european_history.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,398 copying lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_other.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,400 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_global_facts.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,403 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_management.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,405 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_public_relations.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,407 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,410 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_business_ethics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,412 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_accounting.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,414 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_machine_learning.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,417 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,419 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_computer_security.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,422 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_virology.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,425 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,427 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_physics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,430 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,432 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_mathematics.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,435 copying lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_medicine.yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:41,437 copying lm_eval/tasks/mts_dialog/mts_dialog_perplexity.yaml -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T15:52:41,440 copying lm_eval/tasks/mts_dialog/mts_dialog.yaml -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T15:52:41,443 copying lm_eval/tasks/meddialog/meddialog_qsumm.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:41,445 copying lm_eval/tasks/meddialog/meddialog_raw_perplexity.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:41,447 copying lm_eval/tasks/meddialog/meddialog_qsumm_perplexity.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:41,450 copying lm_eval/tasks/meddialog/meddialog_raw_dialogues.yaml -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:41,453 copying lm_eval/tasks/cabbq/cabbq_age.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,455 copying lm_eval/tasks/cabbq/cabbq_nationality.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,457 copying lm_eval/tasks/cabbq/cabbq_ses.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,460 copying lm_eval/tasks/cabbq/cabbq_spanish_region.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,462 copying lm_eval/tasks/cabbq/cabbq.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,464 copying lm_eval/tasks/cabbq/cabbq_gender.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,466 copying lm_eval/tasks/cabbq/cabbq_lgbtqia.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,469 copying lm_eval/tasks/cabbq/cabbq_religion.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,471 copying lm_eval/tasks/cabbq/cabbq_physical_appearance.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,473 copying lm_eval/tasks/cabbq/cabbq_race_ethnicity.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,475 copying lm_eval/tasks/cabbq/cabbq_disability_status.yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:41,477 creating build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,479 copying lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,482 copying lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,484 copying lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,486 copying lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,488 copying lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,491 copying lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,493 copying lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,495 copying lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,497 copying lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,500 copying lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:41,503 copying lm_eval/tasks/babilong/babilong_qa6.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,505 copying lm_eval/tasks/babilong/babilong_qa4.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,507 copying lm_eval/tasks/babilong/babilong_qa15.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,510 copying lm_eval/tasks/babilong/babilong_qa2.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,512 copying lm_eval/tasks/babilong/babilong_qa19.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,514 copying lm_eval/tasks/babilong/babilong.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,517 copying lm_eval/tasks/babilong/babilong_qa1.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,519 copying lm_eval/tasks/babilong/babilong_qa12.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,522 copying lm_eval/tasks/babilong/babilong_qa9.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,524 copying lm_eval/tasks/babilong/babilong_qa20.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,526 copying lm_eval/tasks/babilong/babilong_qa8.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,529 copying lm_eval/tasks/babilong/babilong_qa18.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,532 copying lm_eval/tasks/babilong/babilong_qa7.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,534 copying lm_eval/tasks/babilong/babilong_qa10.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,536 copying lm_eval/tasks/babilong/babilong_qa11.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,539 copying lm_eval/tasks/babilong/babilong_qa3.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,542 copying lm_eval/tasks/babilong/babilong_qa16.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,544 copying lm_eval/tasks/babilong/babilong_longctx.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,547 copying lm_eval/tasks/babilong/babilong_qa17.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,549 copying lm_eval/tasks/babilong/babilong_qa14.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,551 copying lm_eval/tasks/babilong/babilong_qa13.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,554 copying lm_eval/tasks/babilong/babilong_qa5.yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:41,556 copying lm_eval/tasks/careqa/careqa_open_perplexity.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:41,559 copying lm_eval/tasks/careqa/careqa_open.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:41,561 copying lm_eval/tasks/careqa/careqa_es.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:41,563 copying lm_eval/tasks/careqa/careqa_en.yaml -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:41,565 creating build/lib/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T15:52:41,567 copying lm_eval/tasks/mmlu-redux-spanish/mmlu-redux-2.0-spanish.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T15:52:41,569 creating build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,570 copying lm_eval/tasks/mmlu-redux-spanish/generative/_mmlu.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,573 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,575 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,577 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,580 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,582 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,585 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,587 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,590 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,602 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,604 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,606 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_management.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,609 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,611 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,613 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,616 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,619 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,621 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,624 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,626 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,628 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,631 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,633 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,635 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,637 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,640 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,642 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,644 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_virology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,646 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,649 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,651 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,653 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,655 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,658 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,660 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,662 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,665 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,667 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,670 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,673 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,675 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,677 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,680 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,683 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,685 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,688 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,690 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,693 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,695 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,698 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,700 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,702 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,705 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,707 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,709 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,711 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,714 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,716 copying lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:41,719 copying lm_eval/tasks/metabench/metabench_hellaswag_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,721 copying lm_eval/tasks/metabench/metabench_arc_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,723 copying lm_eval/tasks/metabench/metabench_hellaswag.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,726 copying lm_eval/tasks/metabench/metabench_hellaswag_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,728 copying lm_eval/tasks/metabench/metabench.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,730 copying lm_eval/tasks/metabench/metabench_winogrande_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,733 copying lm_eval/tasks/metabench/metabench_mmlu_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,735 copying lm_eval/tasks/metabench/metabench_truthfulqa_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,738 copying lm_eval/tasks/metabench/metabench_mmlu_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,740 copying lm_eval/tasks/metabench/metabench_truthfulqa_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,742 copying lm_eval/tasks/metabench/metabench_gsm8k_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,745 copying lm_eval/tasks/metabench/metabench_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,747 copying lm_eval/tasks/metabench/metabench_hellaswag_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,749 copying lm_eval/tasks/metabench/metabench_mmlu.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,752 copying lm_eval/tasks/metabench/metabench_truthfulqa_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,754 copying lm_eval/tasks/metabench/metabench_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,756 copying lm_eval/tasks/metabench/metabench_arc_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,758 copying lm_eval/tasks/metabench/metabench_winogrande_secondary_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,760 copying lm_eval/tasks/metabench/metabench_winogrande.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,763 copying lm_eval/tasks/metabench/metabench_gsm8k.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,765 copying lm_eval/tasks/metabench/metabench_truthfulqa.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,768 copying lm_eval/tasks/metabench/metabench_arc_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,770 copying lm_eval/tasks/metabench/metabench_winogrande_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,773 copying lm_eval/tasks/metabench/metabench_secondary.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,775 copying lm_eval/tasks/metabench/metabench_arc.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,777 copying lm_eval/tasks/metabench/metabench_mmlu_permute.yaml -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:41,781 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,784 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,786 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,790 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,793 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,796 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,799 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,802 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,805 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,808 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,810 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,813 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,817 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,820 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,822 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,825 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,828 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,831 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,834 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,836 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,839 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,843 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,845 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,848 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,851 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,854 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,857 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,860 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,863 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,866 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,869 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,872 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,875 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,877 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,881 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,885 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,888 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,891 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,894 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,897 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,901 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,904 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,907 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,910 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,913 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,916 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,919 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,921 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,924 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,927 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,930 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,933 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,936 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,939 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,942 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,945 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,948 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,951 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,954 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,957 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,960 copying lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,963 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:41,966 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,969 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,971 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,974 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,977 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,980 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,983 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,986 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,989 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,992 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,994 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,997 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:41,999 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,002 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,005 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,009 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,012 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,015 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,017 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,021 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,024 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,027 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,030 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,033 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,036 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,038 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,041 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,044 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,047 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,049 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,052 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,055 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,058 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,060 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,063 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,066 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,069 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,072 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,075 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,078 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,080 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,083 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,085 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,088 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,090 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,093 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,095 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,098 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,100 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,103 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,105 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,108 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,111 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,113 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,116 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,119 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,121 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,124 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,127 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,129 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,132 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,134 copying lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:42,138 copying lm_eval/tasks/llama3/instruct/arc_challenge/arc_challenge_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T15:52:42,140 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,143 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,145 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,147 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,150 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,152 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,155 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,157 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,160 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,163 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,165 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,168 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,170 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,173 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,175 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,177 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,179 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,182 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,184 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,186 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,189 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,192 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,194 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,197 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,200 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,202 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,205 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,207 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,210 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,212 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,214 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,217 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,220 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,223 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,226 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,229 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,232 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,234 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,237 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,239 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,242 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,244 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,247 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,249 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,252 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,254 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,257 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,260 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,262 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,265 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,267 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,270 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,272 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,275 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,278 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,280 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,283 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,285 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,288 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,290 copying lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,293 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,295 copying lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:42,298 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,301 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,304 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,307 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,310 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,313 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,316 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,320 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,323 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,326 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,329 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,332 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,335 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,337 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,341 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,344 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,348 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,351 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,355 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,358 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,360 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,363 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,366 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,369 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,372 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,375 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,378 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,381 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,384 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,386 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,389 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,392 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,395 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,398 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,401 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,405 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,407 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,411 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,414 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,418 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,421 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,423 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,426 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,429 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,433 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,435 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,438 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,441 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,444 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,447 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,449 copying lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,452 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,454 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,457 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,460 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,463 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,466 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,469 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,471 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,474 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,477 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,480 copying lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:42,482 creating build/lib/lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T15:52:42,483 copying lm_eval/tasks/llama3/instruct/gsm8k/gsm8k.yaml -> build/lib/lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T15:52:42,486 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,489 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,491 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,493 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,496 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,498 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,501 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,503 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,505 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,508 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,510 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,513 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,515 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,517 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,519 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,522 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,525 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,527 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,531 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,533 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,536 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,539 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,541 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,544 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,547 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,549 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,552 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,555 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,557 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,560 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,563 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,566 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,568 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,571 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,573 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,576 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,578 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,581 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,583 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,586 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,589 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,591 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,594 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,596 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,599 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,602 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,604 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,607 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,610 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,612 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,615 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,618 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,621 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,623 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,626 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,629 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,632 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,635 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,638 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,640 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,643 copying lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,645 copying lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:42,648 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,649 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,652 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,654 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,656 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,659 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,661 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,664 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,666 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,669 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,671 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,674 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,677 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,679 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,682 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,684 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,686 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,689 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,692 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,694 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,697 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,700 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,702 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,705 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,707 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,710 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,712 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,714 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,717 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,719 copying lm_eval/tasks/llama3/instruct/mmlu/llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,721 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,723 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,726 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,729 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,731 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,733 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,735 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,738 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,740 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,742 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,745 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,747 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,750 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,752 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,755 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,757 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,760 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,762 copying lm_eval/tasks/llama3/instruct/mmlu/_mmlu_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,765 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,767 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,769 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,772 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,774 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,777 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,779 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,781 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,784 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,786 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,788 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,791 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,793 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,795 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,798 copying lm_eval/tasks/llama3/instruct/mmlu/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:42,801 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,803 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,806 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,809 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,811 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,814 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,817 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,820 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,822 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,825 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,828 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,831 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,834 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,836 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,839 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,841 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,844 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,846 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,849 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,852 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,854 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,856 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,859 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,861 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,864 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,867 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,869 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,871 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,874 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,876 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,879 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,881 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,884 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,886 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,889 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,892 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,894 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,897 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,900 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,902 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,905 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,908 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,911 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,913 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,916 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,918 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,921 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,923 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,926 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,929 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,931 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,933 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,936 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,938 copying lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,941 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,943 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,946 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,948 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,951 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,954 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,956 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,959 copying lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:42,962 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,964 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_business.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,966 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,969 copying lm_eval/tasks/llama3/instruct/mmlu_pro/_mmlu_pro.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,971 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,974 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_health.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,976 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,979 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,982 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,984 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_economics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,986 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,989 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,991 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,993 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,996 copying lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_math.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:42,999 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,002 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,004 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,007 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,009 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,012 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,015 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,018 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,021 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,023 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,026 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,028 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,031 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,034 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,036 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,039 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,042 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,044 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,047 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,050 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,052 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,055 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,057 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,060 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,062 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,065 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,067 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,070 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,072 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,075 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,077 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,080 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,083 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,085 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,088 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,090 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,093 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,096 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,098 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,101 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,104 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,106 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,109 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,111 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,114 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,117 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,119 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,122 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,124 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,127 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,129 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,132 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,134 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,137 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,139 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,142 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,145 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,147 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,150 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,152 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,155 copying lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,157 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:43,159 creating build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,161 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_aging.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,163 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_formal_logic.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,166 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_world_religions.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,168 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,171 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_biology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,173 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_accounting.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,175 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_government_and_politics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,178 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_security_studies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,181 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_international_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,183 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_social_sciences.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,186 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_philosophy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,189 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_management.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,192 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,194 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,196 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,199 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_macroeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,201 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_medical_genetics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,204 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_electrical_engineering.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,206 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,209 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,211 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_machine_learning.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,214 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_chemistry.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,216 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,218 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_microeconomics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,221 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_business_ethics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,223 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_disputes.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,226 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_public_relations.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,230 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_logical_fallacies.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,232 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_virology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,235 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_clinical_knowledge.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,237 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_computer_science.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,240 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_us_foreign_policy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,242 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_scenarios.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,245 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_jurisprudence.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,248 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_medicine.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,251 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_elementary_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,253 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_prehistory.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,256 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_conceptual_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,258 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_nutrition.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,261 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_mathematics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,264 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_stem.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,266 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_european_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,269 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_humanities.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,272 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,275 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_econometrics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,278 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_marketing.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,280 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_other.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,282 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_sociology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,285 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_law.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,287 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_world_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,289 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_abstract_algebra.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,292 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_anatomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,294 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_global_facts.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,296 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_miscellaneous.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,299 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_psychology.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,301 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_astronomy.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,303 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_sexuality.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,306 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_statistics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,308 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_physics.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,311 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_us_history.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,313 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_geography.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,315 copying lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_computer_security.yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:43,318 copying lm_eval/tasks/copal_id/standard.yaml -> build/lib/lm_eval/tasks/copal_id 2026-05-11T15:52:43,321 copying lm_eval/tasks/copal_id/colloquial.yaml -> build/lib/lm_eval/tasks/copal_id 2026-05-11T15:52:43,325 copying lm_eval/tasks/README.md -> build/lib/lm_eval/tasks 2026-05-11T15:52:43,332 copying lm_eval/tasks/noticia/README.md -> build/lib/lm_eval/tasks/noticia 2026-05-11T15:52:43,337 copying lm_eval/tasks/arab_culture/_default_arab_culture_mcq_template_yaml -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:43,340 copying lm_eval/tasks/arab_culture/README.md -> build/lib/lm_eval/tasks/arab_culture 2026-05-11T15:52:43,343 copying lm_eval/tasks/tmmluplus/subject.tsv -> build/lib/lm_eval/tasks/tmmluplus 2026-05-11T15:52:43,345 copying lm_eval/tasks/tmmluplus/README.md -> build/lib/lm_eval/tasks/tmmluplus 2026-05-11T15:52:43,348 copying lm_eval/tasks/tmmluplus/default/_tmmluplus_template_yaml -> build/lib/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:43,352 copying lm_eval/tasks/mimic_repsum/README.md -> build/lib/lm_eval/tasks/mimic_repsum 2026-05-11T15:52:43,356 copying lm_eval/tasks/cnn_dailymail/README.md -> build/lib/lm_eval/tasks/cnn_dailymail 2026-05-11T15:52:43,360 copying lm_eval/tasks/aradice/README.md -> build/lib/lm_eval/tasks/aradice 2026-05-11T15:52:43,364 copying lm_eval/tasks/aradice/ArabicMMLU/LEV/_default_template_yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:43,368 copying lm_eval/tasks/aradice/ArabicMMLU/EGY/_default_template_yaml -> build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:43,384 copying lm_eval/tasks/mmlu-redux/generative/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:43,386 copying lm_eval/tasks/mmlu-redux/generative/README.md -> build/lib/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:43,389 copying lm_eval/tasks/mc_taco/README.md -> build/lib/lm_eval/tasks/mc_taco 2026-05-11T15:52:43,392 copying lm_eval/tasks/humaneval/README.md -> build/lib/lm_eval/tasks/humaneval 2026-05-11T15:52:43,396 copying lm_eval/tasks/multiblimp/_template_yaml -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:43,398 copying lm_eval/tasks/multiblimp/README.md -> build/lib/lm_eval/tasks/multiblimp 2026-05-11T15:52:43,401 copying lm_eval/tasks/commonsense_qa/README.md -> build/lib/lm_eval/tasks/commonsense_qa 2026-05-11T15:52:43,404 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/_var5shots_template_yaml -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:43,406 copying lm_eval/tasks/e2lmc/sciknoweval_mcqa/README.md -> build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:43,409 copying lm_eval/tasks/e2lmc/noor/README.md -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:43,412 copying lm_eval/tasks/e2lmc/noor/_noor_template -> build/lib/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:43,415 copying lm_eval/tasks/e2lmc/mmlu_early_training/README.md -> build/lib/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:43,418 copying lm_eval/tasks/paloma/_paloma_template -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:43,420 copying lm_eval/tasks/paloma/README.md -> build/lib/lm_eval/tasks/paloma 2026-05-11T15:52:43,424 copying lm_eval/tasks/c4/README.md -> build/lib/lm_eval/tasks/c4 2026-05-11T15:52:43,428 copying lm_eval/tasks/mmlu_prox/README.md -> build/lib/lm_eval/tasks/mmlu_prox 2026-05-11T15:52:43,432 copying lm_eval/tasks/mmlu_prox/mr/_mr_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:43,434 copying lm_eval/tasks/mmlu_prox/mr/_mr_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:43,438 copying lm_eval/tasks/mmlu_prox/de/_de_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:43,440 copying lm_eval/tasks/mmlu_prox/de/_de_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:43,443 copying lm_eval/tasks/mmlu_prox/yo/_yo_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:43,447 copying lm_eval/tasks/mmlu_prox/yo/_yo_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:43,450 copying lm_eval/tasks/mmlu_prox/ru/_ru_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:43,452 copying lm_eval/tasks/mmlu_prox/ru/_ru_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:43,456 copying lm_eval/tasks/mmlu_prox/zh/_zh_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:43,459 copying lm_eval/tasks/mmlu_prox/zh/_zh_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:43,462 copying lm_eval/tasks/mmlu_prox/af/_af_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:43,464 copying lm_eval/tasks/mmlu_prox/af/_af_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:43,468 copying lm_eval/tasks/mmlu_prox/th/_th_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:43,470 copying lm_eval/tasks/mmlu_prox/th/_th_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:43,473 copying lm_eval/tasks/mmlu_prox/zu/_zu_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:43,476 copying lm_eval/tasks/mmlu_prox/zu/_zu_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:43,479 copying lm_eval/tasks/mmlu_prox/ne/_ne_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:43,482 copying lm_eval/tasks/mmlu_prox/ne/_ne_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:43,484 copying lm_eval/tasks/mmlu_prox/vi/_vi_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:43,487 copying lm_eval/tasks/mmlu_prox/vi/_vi_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:43,490 copying lm_eval/tasks/mmlu_prox/ko/_ko_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:43,493 copying lm_eval/tasks/mmlu_prox/ko/_ko_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:43,496 copying lm_eval/tasks/mmlu_prox/bn/_bn_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:43,498 copying lm_eval/tasks/mmlu_prox/bn/_bn_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:43,502 copying lm_eval/tasks/mmlu_prox/template/_lang_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/template 2026-05-11T15:52:43,505 copying lm_eval/tasks/mmlu_prox/en/_en_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:43,508 copying lm_eval/tasks/mmlu_prox/en/_en_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:43,511 copying lm_eval/tasks/mmlu_prox/uk/_uk_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:43,513 copying lm_eval/tasks/mmlu_prox/uk/_uk_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:43,517 copying lm_eval/tasks/mmlu_prox/sw/_sw_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:43,520 copying lm_eval/tasks/mmlu_prox/sw/_sw_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:43,523 copying lm_eval/tasks/mmlu_prox/es/_es_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:43,525 copying lm_eval/tasks/mmlu_prox/es/_es_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:43,528 copying lm_eval/tasks/mmlu_prox/te/_te_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:43,531 copying lm_eval/tasks/mmlu_prox/te/_te_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:43,534 copying lm_eval/tasks/mmlu_prox/ja/_ja_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:43,537 copying lm_eval/tasks/mmlu_prox/ja/_ja_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:43,540 copying lm_eval/tasks/mmlu_prox/hi/_hi_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:43,543 copying lm_eval/tasks/mmlu_prox/hi/_hi_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:43,546 copying lm_eval/tasks/mmlu_prox/it/_it_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:43,549 copying lm_eval/tasks/mmlu_prox/it/_it_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:43,552 copying lm_eval/tasks/mmlu_prox/sr/_sr_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:43,555 copying lm_eval/tasks/mmlu_prox/sr/_sr_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:43,558 copying lm_eval/tasks/mmlu_prox/wo/_wo_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:43,560 copying lm_eval/tasks/mmlu_prox/wo/_wo_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:43,564 copying lm_eval/tasks/mmlu_prox/fr/_fr_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:43,566 copying lm_eval/tasks/mmlu_prox/fr/_fr_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:43,569 copying lm_eval/tasks/mmlu_prox/cs/_cs_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:43,571 copying lm_eval/tasks/mmlu_prox/cs/_cs_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:43,574 copying lm_eval/tasks/mmlu_prox/pt/_pt_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:43,577 copying lm_eval/tasks/mmlu_prox/pt/_pt_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:43,580 copying lm_eval/tasks/mmlu_prox/id/_id_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:43,582 copying lm_eval/tasks/mmlu_prox/id/_id_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:43,585 copying lm_eval/tasks/mmlu_prox/ar/_ar_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:43,588 copying lm_eval/tasks/mmlu_prox/ar/_ar_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:43,591 copying lm_eval/tasks/mmlu_prox/ur/_ur_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:43,593 copying lm_eval/tasks/mmlu_prox/ur/_ur_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:43,597 copying lm_eval/tasks/mmlu_prox/hu/_hu_lite_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:43,600 copying lm_eval/tasks/mmlu_prox/hu/_hu_template_yaml -> build/lib/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:43,602 copying lm_eval/tasks/alghafa/piqa_ar/README.md -> build/lib/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T15:52:43,605 copying lm_eval/tasks/alghafa/copa_ar/README.md -> build/lib/lm_eval/tasks/alghafa/copa_ar 2026-05-11T15:52:43,607 copying lm_eval/tasks/graphwalks/README.md -> build/lib/lm_eval/tasks/graphwalks 2026-05-11T15:52:43,610 copying lm_eval/tasks/simple_cooccurrence_bias/README.md -> build/lib/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:43,614 copying lm_eval/tasks/catalan_bench/_cabreu_common_yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:43,616 copying lm_eval/tasks/catalan_bench/_arc_ca_common_yaml -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:43,618 copying lm_eval/tasks/catalan_bench/README.md -> build/lib/lm_eval/tasks/catalan_bench 2026-05-11T15:52:43,623 copying lm_eval/tasks/catalan_bench/phrases_va/_phrases_va_common -> build/lib/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:43,625 copying lm_eval/tasks/catalan_bench/flores_ca/_flores_common_yaml -> build/lib/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:43,629 copying lm_eval/tasks/bangla/README.md -> build/lib/lm_eval/tasks/bangla 2026-05-11T15:52:43,631 copying lm_eval/tasks/winogender/README.md -> build/lib/lm_eval/tasks/winogender 2026-05-11T15:52:43,635 copying lm_eval/tasks/olaph/README.md -> build/lib/lm_eval/tasks/olaph 2026-05-11T15:52:43,638 copying lm_eval/tasks/mediqa_qa2019/README.md -> build/lib/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:43,643 copying lm_eval/tasks/pile_10k/README.md -> build/lib/lm_eval/tasks/pile_10k 2026-05-11T15:52:43,645 copying lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:43,648 copying lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:43,651 copying lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml -> build/lib/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:43,654 copying lm_eval/tasks/okapi/arc_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:43,658 copying lm_eval/tasks/okapi/arc_multilingual/_arc_yaml -> build/lib/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:43,660 copying lm_eval/tasks/okapi/truthfulqa_multilingual/README.md -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:43,662 copying lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:43,664 copying lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml -> build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:43,668 copying lm_eval/tasks/eus_exams/eus_exams -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:43,670 copying lm_eval/tasks/eus_exams/eus_exams_es -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:43,673 copying lm_eval/tasks/eus_exams/README.md -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:43,676 copying lm_eval/tasks/eus_exams/eus_exams_eu -> build/lib/lm_eval/tasks/eus_exams 2026-05-11T15:52:43,678 copying lm_eval/tasks/qasper/README.md -> build/lib/lm_eval/tasks/qasper 2026-05-11T15:52:43,682 copying lm_eval/tasks/aexams/_default_template_yaml -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:43,684 copying lm_eval/tasks/aexams/README.md -> build/lib/lm_eval/tasks/aexams 2026-05-11T15:52:43,687 copying lm_eval/tasks/translation/wmt_common_yaml -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:43,689 copying lm_eval/tasks/translation/README.md -> build/lib/lm_eval/tasks/translation 2026-05-11T15:52:43,692 copying lm_eval/tasks/mutual/README.md -> build/lib/lm_eval/tasks/mutual 2026-05-11T15:52:43,696 copying lm_eval/tasks/kmmlu/README.md -> build/lib/lm_eval/tasks/kmmlu 2026-05-11T15:52:43,698 copying lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:43,701 copying lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:43,704 copying lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:43,706 copying lm_eval/tasks/kmmlu/default/_default_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:43,709 copying lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml -> build/lib/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:43,712 copying lm_eval/tasks/swde/README.md -> build/lib/lm_eval/tasks/swde 2026-05-11T15:52:43,715 copying lm_eval/tasks/lambada_multilingual/README.md -> build/lib/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:43,718 copying lm_eval/tasks/belebele/_default_template_yaml -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:43,721 copying lm_eval/tasks/belebele/README.md -> build/lib/lm_eval/tasks/belebele 2026-05-11T15:52:43,723 copying lm_eval/tasks/winogrande/README.md -> build/lib/lm_eval/tasks/winogrande 2026-05-11T15:52:43,727 copying lm_eval/tasks/chartqa/README.md -> build/lib/lm_eval/tasks/chartqa 2026-05-11T15:52:43,730 copying lm_eval/tasks/crows_pairs/README.md -> build/lib/lm_eval/tasks/crows_pairs 2026-05-11T15:52:43,734 copying lm_eval/tasks/click/README.md -> build/lib/lm_eval/tasks/click 2026-05-11T15:52:43,736 copying lm_eval/tasks/click/click_lang/_default_click_lang_yaml -> build/lib/lm_eval/tasks/click/click_lang 2026-05-11T15:52:43,740 copying lm_eval/tasks/click/click_cul/_default_click_cul_yaml -> build/lib/lm_eval/tasks/click/click_cul 2026-05-11T15:52:43,743 copying lm_eval/tasks/kobest/README.md -> build/lib/lm_eval/tasks/kobest 2026-05-11T15:52:43,746 copying lm_eval/tasks/squad_completion/README.md -> build/lib/lm_eval/tasks/squad_completion 2026-05-11T15:52:43,749 copying lm_eval/tasks/zhoblimp/_template_yaml -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:43,751 copying lm_eval/tasks/zhoblimp/README.md -> build/lib/lm_eval/tasks/zhoblimp 2026-05-11T15:52:43,754 copying lm_eval/tasks/wsc273/README.md -> build/lib/lm_eval/tasks/wsc273 2026-05-11T15:52:43,757 copying lm_eval/tasks/unitxt/unitxt -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:43,760 copying lm_eval/tasks/unitxt/README.md -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:43,762 copying lm_eval/tasks/unitxt/unitxt_multimodal -> build/lib/lm_eval/tasks/unitxt 2026-05-11T15:52:43,765 copying lm_eval/tasks/arabic_leaderboard_light/README.md -> build/lib/lm_eval/tasks/arabic_leaderboard_light 2026-05-11T15:52:43,779 copying lm_eval/tasks/minerva_math/README.md -> build/lib/lm_eval/tasks/minerva_math 2026-05-11T15:52:43,782 copying lm_eval/tasks/hellaswag/README.md -> build/lib/lm_eval/tasks/hellaswag 2026-05-11T15:52:43,785 copying lm_eval/tasks/lambada_multilingual_stablelm/README.md -> build/lib/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:43,789 copying lm_eval/tasks/score/non_greedy.sh -> build/lib/lm_eval/tasks/score 2026-05-11T15:52:43,791 copying lm_eval/tasks/score/NON_GREEDY.md -> build/lib/lm_eval/tasks/score 2026-05-11T15:52:43,793 copying lm_eval/tasks/score/README.md -> build/lib/lm_eval/tasks/score 2026-05-11T15:52:43,796 copying lm_eval/tasks/score/agi_eval/prompt_templates.json -> build/lib/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:43,800 copying lm_eval/tasks/score/math/to_be_fixed_questions.json -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:43,803 copying lm_eval/tasks/score/math/prompt_templates.json -> build/lib/lm_eval/tasks/score/math 2026-05-11T15:52:43,807 copying lm_eval/tasks/score/mmlu_pro/prompt_templates.json -> build/lib/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:52:43,810 copying lm_eval/tasks/meqsum/README.md -> build/lib/lm_eval/tasks/meqsum 2026-05-11T15:52:43,813 copying lm_eval/tasks/bear/README.md -> build/lib/lm_eval/tasks/bear 2026-05-11T15:52:43,816 copying lm_eval/tasks/gsm8k/README.md -> build/lib/lm_eval/tasks/gsm8k 2026-05-11T15:52:43,819 copying lm_eval/tasks/libra/_template_yaml -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:43,821 copying lm_eval/tasks/libra/README.md -> build/lib/lm_eval/tasks/libra 2026-05-11T15:52:43,825 copying lm_eval/tasks/glianorex/README.md -> build/lib/lm_eval/tasks/glianorex 2026-05-11T15:52:43,829 copying lm_eval/tasks/hendrycks_math/README.md -> build/lib/lm_eval/tasks/hendrycks_math 2026-05-11T15:52:43,832 copying lm_eval/tasks/mmlu-pro-plus/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:43,834 copying lm_eval/tasks/mmlu-pro-plus/README.md -> build/lib/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:52:43,838 copying lm_eval/tasks/acpbench/README.md -> build/lib/lm_eval/tasks/acpbench 2026-05-11T15:52:43,841 copying lm_eval/tasks/acpbench/boolq_cot_2shot/_boolq_cot_2shot_yaml -> build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:52:43,844 copying lm_eval/tasks/acpbench/gen_2shot/_gen_yaml_2shot -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:43,846 copying lm_eval/tasks/acpbench/gen_2shot/acp_grammar.lark -> build/lib/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:52:43,849 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/_gen_yaml_2shot -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:43,851 copying lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_grammar.lark -> build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:52:43,854 copying lm_eval/tasks/acpbench/mcq_cot_2shot/_mcq_cot_2shot_yaml -> build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:52:43,857 copying lm_eval/tasks/arab_culture_completion/_default_arab_culture_completion_template_yaml -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:43,861 copying lm_eval/tasks/arab_culture_completion/README.md -> build/lib/lm_eval/tasks/arab_culture_completion 2026-05-11T15:52:43,863 copying lm_eval/tasks/storycloze/README.md -> build/lib/lm_eval/tasks/storycloze 2026-05-11T15:52:43,866 copying lm_eval/tasks/basque_bench/README.md -> build/lib/lm_eval/tasks/basque_bench 2026-05-11T15:52:43,870 copying lm_eval/tasks/basque_bench/flores_eu/_flores_common_yaml -> build/lib/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:52:43,873 copying lm_eval/tasks/darijammlu/README.md -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:43,875 copying lm_eval/tasks/darijammlu/_default_darijammlu_template_yaml -> build/lib/lm_eval/tasks/darijammlu 2026-05-11T15:52:43,878 copying lm_eval/tasks/jsonschema_bench/README.md -> build/lib/lm_eval/tasks/jsonschema_bench 2026-05-11T15:52:43,882 copying lm_eval/tasks/discrim_eval/README.md -> build/lib/lm_eval/tasks/discrim_eval 2026-05-11T15:52:43,885 copying lm_eval/tasks/super_glue/README.md -> build/lib/lm_eval/tasks/super_glue 2026-05-11T15:52:43,894 copying lm_eval/tasks/hrm8k/README.md -> build/lib/lm_eval/tasks/hrm8k 2026-05-11T15:52:43,897 copying lm_eval/tasks/hrm8k/en/_hrm8k_en_yaml -> build/lib/lm_eval/tasks/hrm8k/en 2026-05-11T15:52:43,900 copying lm_eval/tasks/hrm8k/default/_hrm8k_yaml -> build/lib/lm_eval/tasks/hrm8k/default 2026-05-11T15:52:43,903 copying lm_eval/tasks/toxigen/README.md -> build/lib/lm_eval/tasks/toxigen 2026-05-11T15:52:43,906 copying lm_eval/tasks/icelandic_winogrande/README.md -> build/lib/lm_eval/tasks/icelandic_winogrande 2026-05-11T15:52:43,909 copying lm_eval/tasks/darijahellaswag/README.md -> build/lib/lm_eval/tasks/darijahellaswag 2026-05-11T15:52:43,912 copying lm_eval/tasks/gsm8k_platinum/README.md -> build/lib/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:52:43,914 copying lm_eval/tasks/truthfulqa/README.md -> build/lib/lm_eval/tasks/truthfulqa 2026-05-11T15:52:43,917 copying lm_eval/tasks/bbq/README.md -> build/lib/lm_eval/tasks/bbq 2026-05-11T15:52:43,920 copying lm_eval/tasks/bigbench/multiple_choice_template_a_yaml -> build/lib/lm_eval/tasks/bigbench 2026-05-11T15:52:43,923 copying lm_eval/tasks/bigbench/multiple_choice_template_b_yaml -> build/lib/lm_eval/tasks/bigbench 2026-05-11T15:52:43,926 copying lm_eval/tasks/bigbench/README.md -> build/lib/lm_eval/tasks/bigbench 2026-05-11T15:52:43,929 copying lm_eval/tasks/bigbench/generate_until_template_yaml -> build/lib/lm_eval/tasks/bigbench 2026-05-11T15:52:43,931 copying lm_eval/tasks/arabic_leaderboard_complete/README.md -> build/lib/lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T15:52:43,947 copying lm_eval/tasks/pubmedqa/README.md -> build/lib/lm_eval/tasks/pubmedqa 2026-05-11T15:52:43,950 copying lm_eval/tasks/scrolls/README.md -> build/lib/lm_eval/tasks/scrolls 2026-05-11T15:52:43,953 copying lm_eval/tasks/mlqa/mlqa_common_yaml -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:43,956 copying lm_eval/tasks/mlqa/README.md -> build/lib/lm_eval/tasks/mlqa 2026-05-11T15:52:43,959 copying lm_eval/tasks/noreval/README.md -> build/lib/lm_eval/tasks/noreval 2026-05-11T15:52:43,962 copying lm_eval/tasks/noreval/noreval.jpg -> build/lib/lm_eval/tasks/noreval 2026-05-11T15:52:43,975 copying lm_eval/tasks/noreval/tatoeba/_tatoeba_yaml -> build/lib/lm_eval/tasks/noreval/tatoeba 2026-05-11T15:52:43,978 copying lm_eval/tasks/noreval/noropenbookqa/_noropenbookqa_yaml -> build/lib/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T15:52:43,982 copying lm_eval/tasks/noreval/nrk_quiz_qa/_nrk_quiz_qa_yaml -> build/lib/lm_eval/tasks/noreval/nrk_quiz_qa 2026-05-11T15:52:43,986 copying lm_eval/tasks/noreval/norec/_norec_yaml -> build/lib/lm_eval/tasks/noreval/norec 2026-05-11T15:52:43,990 copying lm_eval/tasks/noreval/noridiom/_noridiom_yaml -> build/lib/lm_eval/tasks/noreval/noridiom 2026-05-11T15:52:43,994 copying lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/_nortruthfulqa_mc_yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice 2026-05-11T15:52:43,998 copying lm_eval/tasks/noreval/nortruthfulqa/generation/_nortruthfulqa_gen_yaml -> build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T15:52:44,001 copying lm_eval/tasks/noreval/norsumm/_norsumm_yaml -> build/lib/lm_eval/tasks/noreval/norsumm 2026-05-11T15:52:44,005 copying lm_eval/tasks/noreval/norquad/_norquad_yaml -> build/lib/lm_eval/tasks/noreval/norquad 2026-05-11T15:52:44,009 copying lm_eval/tasks/noreval/norcommonsenseqa/_norcommonsenseqa_yaml -> build/lib/lm_eval/tasks/noreval/norcommonsenseqa 2026-05-11T15:52:44,012 copying lm_eval/tasks/noreval/norbelebele/_norbelebele_yaml -> build/lib/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:52:44,015 copying lm_eval/tasks/noreval/ask_gec/README.md -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:44,018 copying lm_eval/tasks/noreval/ask_gec/_ask_gec_yaml -> build/lib/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:52:44,020 copying lm_eval/tasks/eus_trivia/README.md -> build/lib/lm_eval/tasks/eus_trivia 2026-05-11T15:52:44,024 copying lm_eval/tasks/ulqa/README.md -> build/lib/lm_eval/tasks/ulqa 2026-05-11T15:52:44,027 copying lm_eval/tasks/fda/README.md -> build/lib/lm_eval/tasks/fda 2026-05-11T15:52:44,031 copying lm_eval/tasks/coqa/README.md -> build/lib/lm_eval/tasks/coqa 2026-05-11T15:52:44,034 copying lm_eval/tasks/xnli_eu/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T15:52:44,037 copying lm_eval/tasks/xnli_eu/README.md -> build/lib/lm_eval/tasks/xnli_eu 2026-05-11T15:52:44,044 copying lm_eval/tasks/ruler/README.md -> build/lib/lm_eval/tasks/ruler 2026-05-11T15:52:44,051 copying lm_eval/tasks/japanese_leaderboard/requirements.txt -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:44,055 copying lm_eval/tasks/japanese_leaderboard/README.md -> build/lib/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:52:44,059 copying lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:52:44,062 creating build/lib/lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T15:52:44,064 copying lm_eval/tasks/model_written_evals/winogenerated/winogenerated -> build/lib/lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T15:52:44,067 copying lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:52:44,071 copying lm_eval/tasks/wikitext/README.md -> build/lib/lm_eval/tasks/wikitext 2026-05-11T15:52:44,075 copying lm_eval/tasks/arc/README.md -> build/lib/lm_eval/tasks/arc 2026-05-11T15:52:44,078 copying lm_eval/tasks/swag/README.md -> build/lib/lm_eval/tasks/swag 2026-05-11T15:52:44,083 copying lm_eval/tasks/bbh/README.md -> build/lib/lm_eval/tasks/bbh 2026-05-11T15:52:44,086 copying lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/fewshot 2026-05-11T15:52:44,089 copying lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:52:44,092 copying lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:52:44,096 copying lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/lib/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:52:44,100 copying lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/lib/lm_eval/tasks/csatqa 2026-05-11T15:52:44,104 copying lm_eval/tasks/xnli/xnli_common_yaml -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:44,107 copying lm_eval/tasks/xnli/README.md -> build/lib/lm_eval/tasks/xnli 2026-05-11T15:52:44,112 copying lm_eval/tasks/global_piqa/README.md -> build/lib/lm_eval/tasks/global_piqa 2026-05-11T15:52:44,114 copying lm_eval/tasks/global_piqa/completions/_template -> build/lib/lm_eval/tasks/global_piqa/completions 2026-05-11T15:52:44,118 copying lm_eval/tasks/global_piqa/prompted/_template -> build/lib/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:52:44,122 copying lm_eval/tasks/bhs/_template_yaml -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:44,125 copying lm_eval/tasks/bhs/README.md -> build/lib/lm_eval/tasks/bhs 2026-05-11T15:52:44,128 copying lm_eval/tasks/lambada_cloze/README.md -> build/lib/lm_eval/tasks/lambada_cloze 2026-05-11T15:52:44,132 copying lm_eval/tasks/eus_reading/README.md -> build/lib/lm_eval/tasks/eus_reading 2026-05-11T15:52:44,135 copying lm_eval/tasks/longbench/README.md -> build/lib/lm_eval/tasks/longbench 2026-05-11T15:52:44,142 copying lm_eval/tasks/mmlu/README.md -> build/lib/lm_eval/tasks/mmlu 2026-05-11T15:52:44,145 copying lm_eval/tasks/mmlu/continuation/_continuation_template_yaml -> build/lib/lm_eval/tasks/mmlu/continuation 2026-05-11T15:52:44,148 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:44,154 copying lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:52:44,157 copying lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:52:44,162 copying lm_eval/tasks/mmlu/generative/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/generative 2026-05-11T15:52:44,165 copying lm_eval/tasks/mmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu/default 2026-05-11T15:52:44,168 copying lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:52:44,172 copying lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:52:44,174 copying lm_eval/tasks/headqa/README.md -> build/lib/lm_eval/tasks/headqa 2026-05-11T15:52:44,178 copying lm_eval/tasks/blimp/_template_yaml -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:44,181 copying lm_eval/tasks/blimp/README.md -> build/lib/lm_eval/tasks/blimp 2026-05-11T15:52:44,184 copying lm_eval/tasks/mela/README.md -> build/lib/lm_eval/tasks/mela 2026-05-11T15:52:44,186 copying lm_eval/tasks/mmmu/_template_yaml -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:44,190 copying lm_eval/tasks/mmmu/README.md -> build/lib/lm_eval/tasks/mmmu 2026-05-11T15:52:44,194 copying lm_eval/tasks/histoires_morales/README.md -> build/lib/lm_eval/tasks/histoires_morales 2026-05-11T15:52:44,197 copying lm_eval/tasks/prost/README.md -> build/lib/lm_eval/tasks/prost 2026-05-11T15:52:44,200 copying lm_eval/tasks/mastermind/README.md -> build/lib/lm_eval/tasks/mastermind 2026-05-11T15:52:44,203 copying lm_eval/tasks/anli/README.md -> build/lib/lm_eval/tasks/anli 2026-05-11T15:52:44,206 copying lm_eval/tasks/longbench2/_longbench_common_yaml -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:44,208 copying lm_eval/tasks/longbench2/README.md -> build/lib/lm_eval/tasks/longbench2 2026-05-11T15:52:44,212 copying lm_eval/tasks/kormedmcqa/_template_yaml -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:44,214 copying lm_eval/tasks/kormedmcqa/README.md -> build/lib/lm_eval/tasks/kormedmcqa 2026-05-11T15:52:44,217 copying lm_eval/tasks/afrimmlu/README.md -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T15:52:44,220 copying lm_eval/tasks/afrimmlu/fewshot.sh -> build/lib/lm_eval/tasks/afrimmlu 2026-05-11T15:52:44,223 copying lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:52:44,227 copying lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:52:44,230 copying lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:52:44,233 copying lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:52:44,236 copying lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate -> build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:52:44,239 copying lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:52:44,242 copying lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:52:44,247 copying lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:52:44,251 copying lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:52:44,255 copying lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct -> build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:52:44,259 copying lm_eval/tasks/mbpp/README.md -> build/lib/lm_eval/tasks/mbpp 2026-05-11T15:52:44,262 copying lm_eval/tasks/blimp_nl/_template_yaml -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:44,265 copying lm_eval/tasks/blimp_nl/README.md -> build/lib/lm_eval/tasks/blimp_nl 2026-05-11T15:52:44,268 copying lm_eval/tasks/moral_stories/README.md -> build/lib/lm_eval/tasks/moral_stories 2026-05-11T15:52:44,272 copying lm_eval/tasks/lingoly/README.md -> build/lib/lm_eval/tasks/lingoly 2026-05-11T15:52:44,276 copying lm_eval/tasks/logiqa2/README.md -> build/lib/lm_eval/tasks/logiqa2 2026-05-11T15:52:44,279 copying lm_eval/tasks/fld/README.md -> build/lib/lm_eval/tasks/fld 2026-05-11T15:52:44,283 copying lm_eval/tasks/darija_bench/README.md -> build/lib/lm_eval/tasks/darija_bench 2026-05-11T15:52:44,286 copying lm_eval/tasks/darija_bench/darija_transliteration/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:44,289 copying lm_eval/tasks/darija_bench/darija_transliteration/transliteration_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:52:44,292 copying lm_eval/tasks/darija_bench/darija_sentiment/default_darija_sentiment_template_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:44,295 copying lm_eval/tasks/darija_bench/darija_sentiment/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:52:44,299 copying lm_eval/tasks/darija_bench/darija_translation/flores_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:44,301 copying lm_eval/tasks/darija_bench/darija_translation/doda_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:44,304 copying lm_eval/tasks/darija_bench/darija_translation/seed_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:44,306 copying lm_eval/tasks/darija_bench/darija_translation/madar_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:44,309 copying lm_eval/tasks/darija_bench/darija_translation/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:44,311 copying lm_eval/tasks/darija_bench/darija_translation/translation_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:52:44,315 copying lm_eval/tasks/darija_bench/darija_summarization/README.md -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:52:44,317 copying lm_eval/tasks/darija_bench/darija_summarization/summarization_common_yaml -> build/lib/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:52:44,321 copying lm_eval/tasks/turkishmmlu/README.md -> build/lib/lm_eval/tasks/turkishmmlu 2026-05-11T15:52:44,325 copying lm_eval/tasks/turkishmmlu/config_cot/_turkishmmlu_cot_default_yaml -> build/lib/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:52:44,327 copying lm_eval/tasks/turkishmmlu/config/_turkishmmlu_default_yaml -> build/lib/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:52:44,330 copying lm_eval/tasks/polemo2/README.md -> build/lib/lm_eval/tasks/polemo2 2026-05-11T15:52:44,333 copying lm_eval/tasks/mmlu_pro/_default_template_yaml -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:44,336 copying lm_eval/tasks/mmlu_pro/README.md -> build/lib/lm_eval/tasks/mmlu_pro 2026-05-11T15:52:44,340 copying lm_eval/tasks/leaderboard/README.md -> build/lib/lm_eval/tasks/leaderboard 2026-05-11T15:52:44,343 copying lm_eval/tasks/leaderboard/bbh_mc/_fewshot_template_yaml -> build/lib/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:52:44,346 copying lm_eval/tasks/leaderboard/math/_template_yaml -> build/lib/lm_eval/tasks/leaderboard/math 2026-05-11T15:52:44,354 copying lm_eval/tasks/leaderboard/musr/_template_yaml -> build/lib/lm_eval/tasks/leaderboard/musr 2026-05-11T15:52:44,358 copying lm_eval/tasks/leaderboard/gpqa/_template_yaml -> build/lib/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:52:44,362 copying lm_eval/tasks/sciq/README.md -> build/lib/lm_eval/tasks/sciq 2026-05-11T15:52:44,365 copying lm_eval/tasks/ceval/_default_ceval_yaml -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:44,368 copying lm_eval/tasks/ceval/README.md -> build/lib/lm_eval/tasks/ceval 2026-05-11T15:52:44,371 copying lm_eval/tasks/aclue/_default_template_yaml -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:44,375 copying lm_eval/tasks/aclue/README.md -> build/lib/lm_eval/tasks/aclue 2026-05-11T15:52:44,378 copying lm_eval/tasks/webqs/README.md -> build/lib/lm_eval/tasks/webqs 2026-05-11T15:52:44,381 copying lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:44,384 copying lm_eval/tasks/hendrycks_ethics/README.md -> build/lib/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:52:44,388 copying lm_eval/tasks/afrimgsm/run.sh -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T15:52:44,391 copying lm_eval/tasks/afrimgsm/gen_yaml.sh -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T15:52:44,393 copying lm_eval/tasks/afrimgsm/README.md -> build/lib/lm_eval/tasks/afrimgsm 2026-05-11T15:52:44,397 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:52:44,399 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:52:44,402 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:52:44,405 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:52:44,407 copying lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:52:44,410 copying lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:52:44,413 copying lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:52:44,415 copying lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:52:44,418 copying lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:52:44,421 copying lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:52:44,423 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:52:44,426 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:52:44,429 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:52:44,432 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:52:44,434 copying lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yaml -> build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:52:44,437 copying lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:52:44,440 copying lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:52:44,442 copying lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:52:44,445 copying lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:52:44,448 copying lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yaml -> build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:52:44,451 copying lm_eval/tasks/bertaqa/_bertaqa_template -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:44,453 copying lm_eval/tasks/bertaqa/README.md -> build/lib/lm_eval/tasks/bertaqa 2026-05-11T15:52:44,457 copying lm_eval/tasks/unscramble/README.md -> build/lib/lm_eval/tasks/unscramble 2026-05-11T15:52:44,461 copying lm_eval/tasks/arc_mt/README.md -> build/lib/lm_eval/tasks/arc_mt 2026-05-11T15:52:44,464 copying lm_eval/tasks/openbookqa/README.md -> build/lib/lm_eval/tasks/openbookqa 2026-05-11T15:52:44,468 copying lm_eval/tasks/nq_open/README.md -> build/lib/lm_eval/tasks/nq_open 2026-05-11T15:52:44,472 copying lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:44,475 copying lm_eval/tasks/xwinograd/README.md -> build/lib/lm_eval/tasks/xwinograd 2026-05-11T15:52:44,483 copying lm_eval/tasks/afrixnli/README.md -> build/lib/lm_eval/tasks/afrixnli 2026-05-11T15:52:44,489 copying lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:52:44,494 copying lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:52:44,498 copying lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:52:44,502 copying lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:52:44,506 copying lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:52:44,511 copying lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:52:44,515 copying lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:52:44,519 copying lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:52:44,523 copying lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:52:44,527 copying lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yaml -> build/lib/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:52:44,532 copying lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:52:44,537 copying lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yaml -> build/lib/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:52:44,541 copying lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:52:44,545 copying lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:52:44,551 copying lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml -> build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:52:44,555 copying lm_eval/tasks/siqa/README.md -> build/lib/lm_eval/tasks/siqa 2026-05-11T15:52:44,558 copying lm_eval/tasks/agieval/README.md -> build/lib/lm_eval/tasks/agieval 2026-05-11T15:52:44,562 copying lm_eval/tasks/babi/README.md -> build/lib/lm_eval/tasks/babi 2026-05-11T15:52:44,566 copying lm_eval/tasks/ifeval/README.md -> build/lib/lm_eval/tasks/ifeval 2026-05-11T15:52:44,576 copying lm_eval/tasks/mgsm/gen_yaml.sh -> build/lib/lm_eval/tasks/mgsm 2026-05-11T15:52:44,579 copying lm_eval/tasks/mgsm/README.md -> build/lib/lm_eval/tasks/mgsm 2026-05-11T15:52:44,583 copying lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:52:44,585 copying lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/lib/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:52:44,588 copying lm_eval/tasks/mgsm/direct/direct_yaml -> build/lib/lm_eval/tasks/mgsm/direct 2026-05-11T15:52:44,591 copying lm_eval/tasks/asdiv/README.md -> build/lib/lm_eval/tasks/asdiv 2026-05-11T15:52:44,594 copying lm_eval/tasks/lambada/README.md -> build/lib/lm_eval/tasks/lambada 2026-05-11T15:52:44,597 copying lm_eval/tasks/triviaqa/README.md -> build/lib/lm_eval/tasks/triviaqa 2026-05-11T15:52:44,602 copying lm_eval/tasks/tinyBenchmarks/README.md -> build/lib/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:52:44,605 copying lm_eval/tasks/humaneval_infilling/README.md -> build/lib/lm_eval/tasks/humaneval_infilling 2026-05-11T15:52:44,609 copying lm_eval/tasks/arabicmmlu/_default_arabicmmlu_template_yaml -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:44,612 copying lm_eval/tasks/arabicmmlu/README.md -> build/lib/lm_eval/tasks/arabicmmlu 2026-05-11T15:52:44,616 copying lm_eval/tasks/french_bench/_default_template_yaml -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:44,618 copying lm_eval/tasks/french_bench/README.md -> build/lib/lm_eval/tasks/french_bench 2026-05-11T15:52:44,622 copying lm_eval/tasks/lm_syneval/_template_yaml -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:44,625 copying lm_eval/tasks/lm_syneval/README.md -> build/lib/lm_eval/tasks/lm_syneval 2026-05-11T15:52:44,629 copying lm_eval/tasks/egymmlu/_default_egymmlu_template_yaml -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:44,632 copying lm_eval/tasks/egymmlu/README.md -> build/lib/lm_eval/tasks/egymmlu 2026-05-11T15:52:44,636 copying lm_eval/tasks/global_mmlu/README.md -> build/lib/lm_eval/tasks/global_mmlu 2026-05-11T15:52:44,639 copying lm_eval/tasks/global_mmlu/default/de/_de_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:52:44,643 copying lm_eval/tasks/global_mmlu/default/yo/_yo_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:52:44,646 copying lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:52:44,650 copying lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:52:44,653 copying lm_eval/tasks/global_mmlu/default/bn/_bn_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:52:44,657 copying lm_eval/tasks/global_mmlu/default/en/_en_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:52:44,661 copying lm_eval/tasks/global_mmlu/default/sw/_sw_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:52:44,664 copying lm_eval/tasks/global_mmlu/default/es/_es_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:52:44,668 copying lm_eval/tasks/global_mmlu/default/ja/_ja_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:52:44,671 copying lm_eval/tasks/global_mmlu/default/hi/_hi_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:52:44,675 copying lm_eval/tasks/global_mmlu/default/it/_it_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:52:44,679 copying lm_eval/tasks/global_mmlu/default/fr/_fr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:52:44,682 copying lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:52:44,686 copying lm_eval/tasks/global_mmlu/default/id/_id_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:52:44,689 copying lm_eval/tasks/global_mmlu/default/ar/_ar_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:52:44,693 copying lm_eval/tasks/global_mmlu/full/ha/_ha_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:52:44,697 copying lm_eval/tasks/global_mmlu/full/ky/_ky_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:52:44,700 copying lm_eval/tasks/global_mmlu/full/de/_de_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:52:44,704 copying lm_eval/tasks/global_mmlu/full/so/_so_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:52:44,708 copying lm_eval/tasks/global_mmlu/full/yo/_yo_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:52:44,711 copying lm_eval/tasks/global_mmlu/full/ru/_ru_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:52:44,714 copying lm_eval/tasks/global_mmlu/full/zh/_zh_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:52:44,717 copying lm_eval/tasks/global_mmlu/full/nl/_nl_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:52:44,721 copying lm_eval/tasks/global_mmlu/full/si/_si_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:52:44,724 copying lm_eval/tasks/global_mmlu/full/ne/_ne_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:52:44,728 copying lm_eval/tasks/global_mmlu/full/vi/_vi_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:52:44,731 copying lm_eval/tasks/global_mmlu/full/ig/_ig_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:52:44,735 copying lm_eval/tasks/global_mmlu/full/ro/_ro_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:52:44,738 copying lm_eval/tasks/global_mmlu/full/ko/_ko_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:52:44,742 copying lm_eval/tasks/global_mmlu/full/lt/_lt_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:52:44,746 copying lm_eval/tasks/global_mmlu/full/pl/_pl_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:52:44,749 copying lm_eval/tasks/global_mmlu/full/sn/_sn_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:52:44,753 copying lm_eval/tasks/global_mmlu/full/bn/_bn_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:52:44,756 copying lm_eval/tasks/global_mmlu/full/ny/_ny_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:52:44,760 copying lm_eval/tasks/global_mmlu/full/en/_en_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:52:44,764 copying lm_eval/tasks/global_mmlu/full/uk/_uk_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:52:44,768 copying lm_eval/tasks/global_mmlu/full/sw/_sw_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:52:44,771 copying lm_eval/tasks/global_mmlu/full/es/_es_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:52:44,775 copying lm_eval/tasks/global_mmlu/full/te/_te_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:52:44,778 copying lm_eval/tasks/global_mmlu/full/fil/_fil_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:52:44,782 copying lm_eval/tasks/global_mmlu/full/ja/_ja_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:52:44,785 copying lm_eval/tasks/global_mmlu/full/hi/_hi_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:52:44,788 copying lm_eval/tasks/global_mmlu/full/ms/_ms_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:52:44,791 copying lm_eval/tasks/global_mmlu/full/it/_it_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:52:44,794 copying lm_eval/tasks/global_mmlu/full/sr/_sr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:52:44,798 copying lm_eval/tasks/global_mmlu/full/mg/_mg_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:52:44,801 copying lm_eval/tasks/global_mmlu/full/sv/_sv_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:52:44,804 copying lm_eval/tasks/global_mmlu/full/fr/_fr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:52:44,807 copying lm_eval/tasks/global_mmlu/full/tr/_tr_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:52:44,811 copying lm_eval/tasks/global_mmlu/full/el/_el_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:52:44,813 copying lm_eval/tasks/global_mmlu/full/he/_he_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:52:44,816 copying lm_eval/tasks/global_mmlu/full/cs/_cs_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:52:44,819 copying lm_eval/tasks/global_mmlu/full/pt/_pt_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:52:44,823 copying lm_eval/tasks/global_mmlu/full/id/_id_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:52:44,826 copying lm_eval/tasks/global_mmlu/full/ar/_ar_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:52:44,829 copying lm_eval/tasks/global_mmlu/full/fa/_fa_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:52:44,832 copying lm_eval/tasks/global_mmlu/full/am/_am_template_yaml -> build/lib/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:52:44,836 copying lm_eval/tasks/tmlu/subject.tsv -> build/lib/lm_eval/tasks/tmlu 2026-05-11T15:52:44,838 copying lm_eval/tasks/tmlu/README.md -> build/lib/lm_eval/tasks/tmlu 2026-05-11T15:52:44,841 copying lm_eval/tasks/tmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/tmlu/default 2026-05-11T15:52:44,844 copying lm_eval/tasks/drop/README.md -> build/lib/lm_eval/tasks/drop 2026-05-11T15:52:44,848 copying lm_eval/tasks/gsm_plus/README.md -> build/lib/lm_eval/tasks/gsm_plus 2026-05-11T15:52:44,850 copying lm_eval/tasks/cmmlu/_default_template_yaml -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:44,853 copying lm_eval/tasks/cmmlu/README.md -> build/lib/lm_eval/tasks/cmmlu 2026-05-11T15:52:44,856 copying lm_eval/tasks/logiqa/README.md -> build/lib/lm_eval/tasks/logiqa 2026-05-11T15:52:44,860 copying lm_eval/tasks/race/README.md -> build/lib/lm_eval/tasks/race 2026-05-11T15:52:44,862 copying lm_eval/tasks/spanish_bench/README.md -> build/lib/lm_eval/tasks/spanish_bench 2026-05-11T15:52:44,866 copying lm_eval/tasks/spanish_bench/phrases_es/_phrases_es_common -> build/lib/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:52:44,868 copying lm_eval/tasks/spanish_bench/flores_es/_flores_common_yaml -> build/lib/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:52:44,872 copying lm_eval/tasks/include/README.md -> build/lib/lm_eval/tasks/include 2026-05-11T15:52:44,874 copying lm_eval/tasks/include/few_shot_en/Indonesian/_indonesian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:52:44,877 copying lm_eval/tasks/include/few_shot_en/Bulgarian/_bulgarian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:52:44,881 copying lm_eval/tasks/include/few_shot_en/Bengali/_bengali_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:52:44,884 copying lm_eval/tasks/include/few_shot_en/Turkish/_turkish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:52:44,887 copying lm_eval/tasks/include/few_shot_en/Belarusian/_belarusian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:52:44,890 copying lm_eval/tasks/include/few_shot_en/Greek/_greek_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:52:44,893 copying lm_eval/tasks/include/few_shot_en/Hungarian/_hungarian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:52:44,897 copying lm_eval/tasks/include/few_shot_en/Dutch/_dutch_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:52:44,900 copying lm_eval/tasks/include/few_shot_en/Korean/_korean_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:52:44,903 copying lm_eval/tasks/include/few_shot_en/German/_german_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:52:44,907 copying lm_eval/tasks/include/few_shot_en/Polish/_polish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:52:44,910 copying lm_eval/tasks/include/few_shot_en/Urdu/_urdu_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:52:44,913 copying lm_eval/tasks/include/few_shot_en/North Macedonian/_north macedonian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:52:44,916 copying lm_eval/tasks/include/few_shot_en/Ukrainian/_ukrainian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:52:44,919 copying lm_eval/tasks/include/few_shot_en/Chinese/_chinese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:52:44,923 copying lm_eval/tasks/include/few_shot_en/Portuguese/_portuguese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:52:44,926 copying lm_eval/tasks/include/few_shot_en/Georgian/_georgian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:52:44,929 copying lm_eval/tasks/include/few_shot_en/French/_french_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:52:44,932 copying lm_eval/tasks/include/few_shot_en/Spanish/_spanish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:52:44,935 copying lm_eval/tasks/include/few_shot_en/Serbian/_serbian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:52:44,939 copying lm_eval/tasks/include/few_shot_en/Malay/_malay_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:52:44,942 copying lm_eval/tasks/include/few_shot_en/Arabic/_arabic_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:52:44,945 copying lm_eval/tasks/include/few_shot_en/Vietnamese/_vietnamese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:52:44,948 copying lm_eval/tasks/include/few_shot_en/Russian/_russian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:52:44,952 copying lm_eval/tasks/include/few_shot_en/Japanese/_japanese_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:52:44,955 copying lm_eval/tasks/include/few_shot_en/Azerbaijani/_azerbaijani_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:52:44,958 copying lm_eval/tasks/include/few_shot_en/Persian/_persian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:52:44,961 copying lm_eval/tasks/include/few_shot_en/Albanian/_albanian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:52:44,965 copying lm_eval/tasks/include/few_shot_en/Croatian/_croatian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:52:44,968 copying lm_eval/tasks/include/few_shot_en/Nepali/_nepali_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:52:44,971 copying lm_eval/tasks/include/few_shot_en/Italian/_italian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:52:44,975 copying lm_eval/tasks/include/few_shot_en/Tamil/_tamil_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:52:44,978 copying lm_eval/tasks/include/few_shot_en/Basque/_basque_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:52:44,981 copying lm_eval/tasks/include/few_shot_en/Hebrew/_hebrew_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:52:44,984 copying lm_eval/tasks/include/few_shot_en/Estonian/_estonian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:52:44,987 copying lm_eval/tasks/include/few_shot_en/Telugu/_telugu_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:52:44,990 copying lm_eval/tasks/include/few_shot_en/Malayalam/_malayalam_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:52:44,994 copying lm_eval/tasks/include/few_shot_en/Kazakh/_kazakh_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:52:44,997 copying lm_eval/tasks/include/few_shot_en/Uzbek/_uzbek_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:52:45,000 copying lm_eval/tasks/include/few_shot_en/Armenian/_armenian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:52:45,003 copying lm_eval/tasks/include/few_shot_en/Finnish/_finnish_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:52:45,006 copying lm_eval/tasks/include/few_shot_en/Tagalog/_tagalog_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:52:45,009 copying lm_eval/tasks/include/few_shot_en/Lithuanian/_lithuanian_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:52:45,012 copying lm_eval/tasks/include/few_shot_en/Hindi/_hindi_few_shot_en_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:52:45,015 copying lm_eval/tasks/include/few_shot_og/Indonesian/_indonesian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:52:45,018 copying lm_eval/tasks/include/few_shot_og/Bulgarian/_bulgarian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:52:45,021 copying lm_eval/tasks/include/few_shot_og/Bengali/_bengali_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:52:45,024 copying lm_eval/tasks/include/few_shot_og/Turkish/_turkish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:52:45,027 copying lm_eval/tasks/include/few_shot_og/Belarusian/_belarusian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:52:45,030 copying lm_eval/tasks/include/few_shot_og/Greek/_greek_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:52:45,033 copying lm_eval/tasks/include/few_shot_og/Hungarian/_hungarian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:52:45,036 copying lm_eval/tasks/include/few_shot_og/Dutch/_dutch_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:52:45,039 copying lm_eval/tasks/include/few_shot_og/Korean/_korean_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:52:45,042 copying lm_eval/tasks/include/few_shot_og/German/_german_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:52:45,045 copying lm_eval/tasks/include/few_shot_og/Polish/_polish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:52:45,048 copying lm_eval/tasks/include/few_shot_og/Urdu/_urdu_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:52:45,052 copying lm_eval/tasks/include/few_shot_og/North Macedonian/_north macedonian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:52:45,055 copying lm_eval/tasks/include/few_shot_og/Ukrainian/_ukrainian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:52:45,058 copying lm_eval/tasks/include/few_shot_og/Chinese/_chinese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:52:45,061 copying lm_eval/tasks/include/few_shot_og/Portuguese/_portuguese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:52:45,065 copying lm_eval/tasks/include/few_shot_og/Georgian/_georgian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:52:45,067 copying lm_eval/tasks/include/few_shot_og/French/_french_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:52:45,070 copying lm_eval/tasks/include/few_shot_og/Spanish/_spanish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:52:45,073 copying lm_eval/tasks/include/few_shot_og/Serbian/_serbian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:52:45,076 copying lm_eval/tasks/include/few_shot_og/Malay/_malay_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:52:45,079 copying lm_eval/tasks/include/few_shot_og/Arabic/_arabic_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:52:45,083 copying lm_eval/tasks/include/few_shot_og/Vietnamese/_vietnamese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:52:45,086 copying lm_eval/tasks/include/few_shot_og/Russian/_russian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:52:45,089 copying lm_eval/tasks/include/few_shot_og/Japanese/_japanese_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:52:45,092 copying lm_eval/tasks/include/few_shot_og/Azerbaijani/_azerbaijani_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:52:45,095 copying lm_eval/tasks/include/few_shot_og/Persian/_persian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:52:45,098 copying lm_eval/tasks/include/few_shot_og/Albanian/_albanian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:52:45,101 copying lm_eval/tasks/include/few_shot_og/Croatian/_croatian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:52:45,104 copying lm_eval/tasks/include/few_shot_og/Nepali/_nepali_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:52:45,107 copying lm_eval/tasks/include/few_shot_og/Italian/_italian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:52:45,110 copying lm_eval/tasks/include/few_shot_og/Tamil/_tamil_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:52:45,114 copying lm_eval/tasks/include/few_shot_og/Basque/_basque_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:52:45,117 copying lm_eval/tasks/include/few_shot_og/Hebrew/_hebrew_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:52:45,120 copying lm_eval/tasks/include/few_shot_og/Estonian/_estonian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:52:45,123 copying lm_eval/tasks/include/few_shot_og/Telugu/_telugu_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:52:45,126 copying lm_eval/tasks/include/few_shot_og/Malayalam/_malayalam_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:52:45,129 copying lm_eval/tasks/include/few_shot_og/Kazakh/_kazakh_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:52:45,132 copying lm_eval/tasks/include/few_shot_og/Uzbek/_uzbek_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:52:45,135 copying lm_eval/tasks/include/few_shot_og/Armenian/_armenian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:52:45,138 copying lm_eval/tasks/include/few_shot_og/Finnish/_finnish_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:52:45,141 copying lm_eval/tasks/include/few_shot_og/Tagalog/_tagalog_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:52:45,144 copying lm_eval/tasks/include/few_shot_og/Lithuanian/_lithuanian_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:52:45,146 copying lm_eval/tasks/include/few_shot_og/Hindi/_hindi_few_shot_og_template_yaml -> build/lib/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:52:45,150 copying lm_eval/tasks/include/default/Indonesian/_indonesian_template_yaml -> build/lib/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:52:45,153 copying lm_eval/tasks/include/default/Bulgarian/_bulgarian_template_yaml -> build/lib/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:52:45,156 copying lm_eval/tasks/include/default/Bengali/_bengali_template_yaml -> build/lib/lm_eval/tasks/include/default/Bengali 2026-05-11T15:52:45,159 copying lm_eval/tasks/include/default/Turkish/_turkish_template_yaml -> build/lib/lm_eval/tasks/include/default/Turkish 2026-05-11T15:52:45,162 copying lm_eval/tasks/include/default/Belarusian/_belarusian_template_yaml -> build/lib/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:52:45,165 copying lm_eval/tasks/include/default/Greek/_greek_template_yaml -> build/lib/lm_eval/tasks/include/default/Greek 2026-05-11T15:52:45,168 copying lm_eval/tasks/include/default/Hungarian/_hungarian_template_yaml -> build/lib/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:52:45,171 copying lm_eval/tasks/include/default/Dutch/_dutch_template_yaml -> build/lib/lm_eval/tasks/include/default/Dutch 2026-05-11T15:52:45,174 copying lm_eval/tasks/include/default/Korean/_korean_template_yaml -> build/lib/lm_eval/tasks/include/default/Korean 2026-05-11T15:52:45,177 copying lm_eval/tasks/include/default/German/_german_template_yaml -> build/lib/lm_eval/tasks/include/default/German 2026-05-11T15:52:45,180 copying lm_eval/tasks/include/default/Polish/_polish_template_yaml -> build/lib/lm_eval/tasks/include/default/Polish 2026-05-11T15:52:45,183 copying lm_eval/tasks/include/default/Urdu/_urdu_template_yaml -> build/lib/lm_eval/tasks/include/default/Urdu 2026-05-11T15:52:45,186 copying lm_eval/tasks/include/default/North Macedonian/_north macedonian_template_yaml -> build/lib/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:52:45,190 copying lm_eval/tasks/include/default/Ukrainian/_ukrainian_template_yaml -> build/lib/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:52:45,193 copying lm_eval/tasks/include/default/Chinese/_chinese_template_yaml -> build/lib/lm_eval/tasks/include/default/Chinese 2026-05-11T15:52:45,197 copying lm_eval/tasks/include/default/Portuguese/_portuguese_template_yaml -> build/lib/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:52:45,200 copying lm_eval/tasks/include/default/Georgian/_georgian_template_yaml -> build/lib/lm_eval/tasks/include/default/Georgian 2026-05-11T15:52:45,203 copying lm_eval/tasks/include/default/French/_french_template_yaml -> build/lib/lm_eval/tasks/include/default/French 2026-05-11T15:52:45,206 copying lm_eval/tasks/include/default/Spanish/_spanish_template_yaml -> build/lib/lm_eval/tasks/include/default/Spanish 2026-05-11T15:52:45,210 copying lm_eval/tasks/include/default/Serbian/_serbian_template_yaml -> build/lib/lm_eval/tasks/include/default/Serbian 2026-05-11T15:52:45,212 copying lm_eval/tasks/include/default/Malay/_malay_template_yaml -> build/lib/lm_eval/tasks/include/default/Malay 2026-05-11T15:52:45,215 copying lm_eval/tasks/include/default/Arabic/_arabic_template_yaml -> build/lib/lm_eval/tasks/include/default/Arabic 2026-05-11T15:52:45,218 copying lm_eval/tasks/include/default/Vietnamese/_vietnamese_template_yaml -> build/lib/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:52:45,221 copying lm_eval/tasks/include/default/Russian/_russian_template_yaml -> build/lib/lm_eval/tasks/include/default/Russian 2026-05-11T15:52:45,224 copying lm_eval/tasks/include/default/Japanese/_japanese_template_yaml -> build/lib/lm_eval/tasks/include/default/Japanese 2026-05-11T15:52:45,226 copying lm_eval/tasks/include/default/Azerbaijani/_azerbaijani_template_yaml -> build/lib/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:52:45,229 copying lm_eval/tasks/include/default/Persian/_persian_template_yaml -> build/lib/lm_eval/tasks/include/default/Persian 2026-05-11T15:52:45,232 copying lm_eval/tasks/include/default/Albanian/_albanian_template_yaml -> build/lib/lm_eval/tasks/include/default/Albanian 2026-05-11T15:52:45,235 copying lm_eval/tasks/include/default/Croatian/_croatian_template_yaml -> build/lib/lm_eval/tasks/include/default/Croatian 2026-05-11T15:52:45,238 copying lm_eval/tasks/include/default/Nepali/_nepali_template_yaml -> build/lib/lm_eval/tasks/include/default/Nepali 2026-05-11T15:52:45,241 copying lm_eval/tasks/include/default/Italian/_italian_template_yaml -> build/lib/lm_eval/tasks/include/default/Italian 2026-05-11T15:52:45,244 copying lm_eval/tasks/include/default/Tamil/_tamil_template_yaml -> build/lib/lm_eval/tasks/include/default/Tamil 2026-05-11T15:52:45,248 copying lm_eval/tasks/include/default/Basque/_basque_template_yaml -> build/lib/lm_eval/tasks/include/default/Basque 2026-05-11T15:52:45,251 copying lm_eval/tasks/include/default/Hebrew/_hebrew_template_yaml -> build/lib/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:52:45,254 copying lm_eval/tasks/include/default/Estonian/_estonian_template_yaml -> build/lib/lm_eval/tasks/include/default/Estonian 2026-05-11T15:52:45,257 copying lm_eval/tasks/include/default/Telugu/_telugu_template_yaml -> build/lib/lm_eval/tasks/include/default/Telugu 2026-05-11T15:52:45,259 copying lm_eval/tasks/include/default/Malayalam/_malayalam_template_yaml -> build/lib/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:52:45,263 copying lm_eval/tasks/include/default/Kazakh/_kazakh_template_yaml -> build/lib/lm_eval/tasks/include/default/Kazakh 2026-05-11T15:52:45,267 copying lm_eval/tasks/include/default/Uzbek/_uzbek_template_yaml -> build/lib/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:52:45,271 copying lm_eval/tasks/include/default/Armenian/_armenian_template_yaml -> build/lib/lm_eval/tasks/include/default/Armenian 2026-05-11T15:52:45,275 copying lm_eval/tasks/include/default/Finnish/_finnish_template_yaml -> build/lib/lm_eval/tasks/include/default/Finnish 2026-05-11T15:52:45,279 copying lm_eval/tasks/include/default/Tagalog/_tagalog_template_yaml -> build/lib/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:52:45,283 copying lm_eval/tasks/include/default/Lithuanian/_lithuanian_template_yaml -> build/lib/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:52:45,287 copying lm_eval/tasks/include/default/Hindi/_hindi_template_yaml -> build/lib/lm_eval/tasks/include/default/Hindi 2026-05-11T15:52:45,291 copying lm_eval/tasks/slr_bench/README.md -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:45,294 copying lm_eval/tasks/slr_bench/slr_bench_common_yaml -> build/lib/lm_eval/tasks/slr_bench 2026-05-11T15:52:45,298 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_common -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:45,301 copying lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc_common -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:45,304 copying lm_eval/tasks/truthfulqa-multi/README.md -> build/lib/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:52:45,308 copying lm_eval/tasks/aime/README.md -> build/lib/lm_eval/tasks/aime 2026-05-11T15:52:45,311 copying lm_eval/tasks/eus_proficiency/README.md -> build/lib/lm_eval/tasks/eus_proficiency 2026-05-11T15:52:45,315 copying lm_eval/tasks/mmlusr/README.md -> build/lib/lm_eval/tasks/mmlusr 2026-05-11T15:52:45,319 copying lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml -> build/lib/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:52:45,322 copying lm_eval/tasks/mmlusr/answer_only/_mmlusr_a_yml -> build/lib/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:52:45,326 copying lm_eval/tasks/mmlusr/question_and_answer/_mmlusr_qna_yml -> build/lib/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:52:45,331 copying lm_eval/tasks/code_x_glue/code-text/_default_template_yaml -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:45,334 copying lm_eval/tasks/code_x_glue/code-text/README.md -> build/lib/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:52:45,339 copying lm_eval/tasks/haerae/_default_haerae_yaml -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:45,342 copying lm_eval/tasks/haerae/README.md -> build/lib/lm_eval/tasks/haerae 2026-05-11T15:52:45,346 copying lm_eval/tasks/squadv2/README.md -> build/lib/lm_eval/tasks/squadv2 2026-05-11T15:52:45,349 copying lm_eval/tasks/galician_bench/README.md -> build/lib/lm_eval/tasks/galician_bench 2026-05-11T15:52:45,354 copying lm_eval/tasks/galician_bench/flores_gl/_flores_common_yaml -> build/lib/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:52:45,357 copying lm_eval/tasks/piqa/README.md -> build/lib/lm_eval/tasks/piqa 2026-05-11T15:52:45,360 copying lm_eval/tasks/eq_bench/README.md -> build/lib/lm_eval/tasks/eq_bench 2026-05-11T15:52:45,364 copying lm_eval/tasks/portuguese_bench/README.md -> build/lib/lm_eval/tasks/portuguese_bench 2026-05-11T15:52:45,368 copying lm_eval/tasks/portuguese_bench/flores_pt/_flores_common_yaml -> build/lib/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:52:45,370 copying lm_eval/tasks/basqueglue/README.md -> build/lib/lm_eval/tasks/basqueglue 2026-05-11T15:52:45,374 copying lm_eval/tasks/inverse_scaling/_some_results -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:45,376 copying lm_eval/tasks/inverse_scaling/_inverse_scaling_mc_yaml -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:45,379 copying lm_eval/tasks/inverse_scaling/README.md -> build/lib/lm_eval/tasks/inverse_scaling 2026-05-11T15:52:45,382 copying lm_eval/tasks/kbl/README.md -> build/lib/lm_eval/tasks/kbl 2026-05-11T15:52:45,385 copying lm_eval/tasks/kbl/reasoning/_kbl_reasoning_yaml -> build/lib/lm_eval/tasks/kbl/reasoning 2026-05-11T15:52:45,387 copying lm_eval/tasks/kbl/knowledge/_kbl_knowledge_yaml -> build/lib/lm_eval/tasks/kbl/knowledge 2026-05-11T15:52:45,390 copying lm_eval/tasks/kbl/bar_exam/responsibility/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:52:45,392 copying lm_eval/tasks/kbl/bar_exam/criminal/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:52:45,394 copying lm_eval/tasks/kbl/bar_exam/civil/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:52:45,397 copying lm_eval/tasks/kbl/bar_exam/public/_base_em_yaml -> build/lib/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:52:45,399 copying lm_eval/tasks/egyhellaswag/README.md -> build/lib/lm_eval/tasks/egyhellaswag 2026-05-11T15:52:45,402 copying lm_eval/tasks/wmt2016/README.md -> build/lib/lm_eval/tasks/wmt2016 2026-05-11T15:52:45,405 copying lm_eval/tasks/mathqa/README.md -> build/lib/lm_eval/tasks/mathqa 2026-05-11T15:52:45,408 copying lm_eval/tasks/qa4mre/README.md -> build/lib/lm_eval/tasks/qa4mre 2026-05-11T15:52:45,412 copying lm_eval/tasks/pisa/_template_yaml -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:45,414 copying lm_eval/tasks/pisa/README.md -> build/lib/lm_eval/tasks/pisa 2026-05-11T15:52:45,417 copying lm_eval/tasks/xstorycloze/README.md -> build/lib/lm_eval/tasks/xstorycloze 2026-05-11T15:52:45,420 copying lm_eval/tasks/esbbq/README.md -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:45,422 copying lm_eval/tasks/esbbq/_esbbq_common_yaml -> build/lib/lm_eval/tasks/esbbq 2026-05-11T15:52:45,426 copying lm_eval/tasks/benchmarks/README.md -> build/lib/lm_eval/tasks/benchmarks 2026-05-11T15:52:45,428 copying lm_eval/tasks/benchmarks/multimedqa/README.md -> build/lib/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T15:52:45,431 copying lm_eval/tasks/benchmarks/flan/_held_in_template_yaml -> build/lib/lm_eval/tasks/benchmarks/flan 2026-05-11T15:52:45,433 copying lm_eval/tasks/med_concepts_qa/_default_template_yaml -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:45,438 copying lm_eval/tasks/med_concepts_qa/README.md -> build/lib/lm_eval/tasks/med_concepts_qa 2026-05-11T15:52:45,440 copying lm_eval/tasks/groundcocoa/README.md -> build/lib/lm_eval/tasks/groundcocoa 2026-05-11T15:52:45,444 copying lm_eval/tasks/evalita_llm/_sum_template_fp-small_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,446 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,448 copying lm_eval/tasks/evalita_llm/_ner_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,450 copying lm_eval/tasks/evalita_llm/_hs_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,453 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,456 copying lm_eval/tasks/evalita_llm/_wic_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,459 copying lm_eval/tasks/evalita_llm/_ls_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,463 copying lm_eval/tasks/evalita_llm/_re_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,466 copying lm_eval/tasks/evalita_llm/_faq_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,469 copying lm_eval/tasks/evalita_llm/_sum_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,471 copying lm_eval/tasks/evalita_llm/_sa_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,473 copying lm_eval/tasks/evalita_llm/_sa_template_v2_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,475 copying lm_eval/tasks/evalita_llm/_te_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,477 copying lm_eval/tasks/evalita_llm/_sum_template_fp_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,480 copying lm_eval/tasks/evalita_llm/README.md -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,482 copying lm_eval/tasks/evalita_llm/_at_template_yaml -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,485 copying lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg -> build/lib/lm_eval/tasks/evalita_llm 2026-05-11T15:52:45,488 copying lm_eval/tasks/medtext/README.md -> build/lib/lm_eval/tasks/medtext 2026-05-11T15:52:45,492 copying lm_eval/tasks/wmdp/_default_template_yaml -> build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:45,494 copying lm_eval/tasks/wmdp/README.md -> build/lib/lm_eval/tasks/wmdp 2026-05-11T15:52:45,496 copying lm_eval/tasks/xquad/xquad_common_yaml -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:45,498 copying lm_eval/tasks/xquad/README.md -> build/lib/lm_eval/tasks/xquad 2026-05-11T15:52:45,502 copying lm_eval/tasks/paws-x/pawsx_template_yaml -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:45,504 copying lm_eval/tasks/paws-x/README.md -> build/lib/lm_eval/tasks/paws-x 2026-05-11T15:52:45,508 copying lm_eval/tasks/pile/README.md -> build/lib/lm_eval/tasks/pile 2026-05-11T15:52:45,510 copying lm_eval/tasks/xcopa/README.md -> build/lib/lm_eval/tasks/xcopa 2026-05-11T15:52:45,514 copying lm_eval/tasks/turblimp/_template_yaml -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:45,516 copying lm_eval/tasks/turblimp/README.md -> build/lib/lm_eval/tasks/turblimp 2026-05-11T15:52:45,519 copying lm_eval/tasks/glue/README.md -> build/lib/lm_eval/tasks/glue 2026-05-11T15:52:45,523 copying lm_eval/tasks/gpqa/README.md -> build/lib/lm_eval/tasks/gpqa 2026-05-11T15:52:45,527 copying lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:52:45,529 copying lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:52:45,533 copying lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml -> build/lib/lm_eval/tasks/gpqa/generative 2026-05-11T15:52:45,537 copying lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml -> build/lib/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:52:45,542 copying lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml -> build/lib/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:52:45,545 copying lm_eval/tasks/afrobench/README.md -> build/lib/lm_eval/tasks/afrobench 2026-05-11T15:52:45,548 copying lm_eval/tasks/afrobench/naijarc/README.md -> build/lib/lm_eval/tasks/afrobench/naijarc 2026-05-11T15:52:45,552 copying lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:52:45,555 copying lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:52:45,558 copying lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:52:45,561 copying lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:52:45,564 copying lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc -> build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:52:45,567 copying lm_eval/tasks/afrobench/sib/README.md -> build/lib/lm_eval/tasks/afrobench/sib 2026-05-11T15:52:45,571 copying lm_eval/tasks/afrobench/sib/prompt_2/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:52:45,574 copying lm_eval/tasks/afrobench/sib/prompt_1/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:52:45,578 copying lm_eval/tasks/afrobench/sib/prompt_5/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:52:45,582 copying lm_eval/tasks/afrobench/sib/prompt_3/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:52:45,585 copying lm_eval/tasks/afrobench/sib/prompt_4/sib -> build/lib/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:52:45,588 copying lm_eval/tasks/afrobench/masakhanews/README.md -> build/lib/lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:52:45,592 copying lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:52:45,595 copying lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:52:45,599 copying lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:52:45,602 copying lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:52:45,606 copying lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews -> build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:52:45,610 copying lm_eval/tasks/afrobench/belebele/README.md -> build/lib/lm_eval/tasks/afrobench/belebele 2026-05-11T15:52:45,614 copying lm_eval/tasks/afrobench/belebele/prompt_2/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:52:45,617 copying lm_eval/tasks/afrobench/belebele/prompt_1/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:52:45,619 copying lm_eval/tasks/afrobench/belebele/prompt_5/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:52:45,622 copying lm_eval/tasks/afrobench/belebele/prompt_3/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:52:45,624 copying lm_eval/tasks/afrobench/belebele/prompt_4/belebele -> build/lib/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:52:45,628 copying lm_eval/tasks/afrobench/masakhapos/README.md -> build/lib/lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:52:45,631 copying lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:52:45,634 copying lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:52:45,637 copying lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:52:45,640 copying lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:52:45,643 copying lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yaml -> build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:52:45,647 copying lm_eval/tasks/afrobench/salt/README.md -> build/lib/lm_eval/tasks/afrobench/salt 2026-05-11T15:52:45,650 copying lm_eval/tasks/afrobench/salt/prompt_2/salt -> build/lib/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:52:45,652 copying lm_eval/tasks/afrobench/salt/prompt_1/salt -> build/lib/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:52:45,654 copying lm_eval/tasks/afrobench/salt/prompt_3/salt -> build/lib/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:52:45,657 copying lm_eval/tasks/afrobench/ntrex/README.md -> build/lib/lm_eval/tasks/afrobench/ntrex 2026-05-11T15:52:45,660 copying lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:52:45,663 copying lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:52:45,665 copying lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:52:45,668 copying lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:52:45,670 copying lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:52:45,673 copying lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex -> build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:52:45,676 copying lm_eval/tasks/afrobench/flores/README.md -> build/lib/lm_eval/tasks/afrobench/flores 2026-05-11T15:52:45,679 copying lm_eval/tasks/afrobench/flores/prompt_2/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2 2026-05-11T15:52:45,682 copying lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:52:45,684 copying lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:52:45,687 copying lm_eval/tasks/afrobench/flores/prompt_1/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1 2026-05-11T15:52:45,690 copying lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:52:45,693 copying lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:52:45,695 copying lm_eval/tasks/afrobench/flores/prompt_3/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3 2026-05-11T15:52:45,698 copying lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:52:45,701 copying lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores -> build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:52:45,704 copying lm_eval/tasks/afrobench/injongointent/README.md -> build/lib/lm_eval/tasks/afrobench/injongointent 2026-05-11T15:52:45,707 copying lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:52:45,710 copying lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:52:45,713 copying lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:52:45,716 copying lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:52:45,720 copying lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent -> build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:52:45,723 copying lm_eval/tasks/afrobench/nollysenti/README.md -> build/lib/lm_eval/tasks/afrobench/nollysenti 2026-05-11T15:52:45,725 copying lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:52:45,728 copying lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:52:45,732 copying lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:52:45,735 copying lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:52:45,738 copying lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti -> build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:52:45,741 copying lm_eval/tasks/afrobench/afrisenti/README.md -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:52:45,743 copying lm_eval/tasks/afrobench/afrisenti/fewshot.sh -> build/lib/lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:52:45,747 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/run.sh -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:45,749 copying lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:52:45,753 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/run.sh -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:45,755 copying lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:52:45,759 copying lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:52:45,763 copying lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:52:45,767 copying lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti -> build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:52:45,772 copying lm_eval/tasks/afrobench/adr/README.md -> build/lib/lm_eval/tasks/afrobench/adr 2026-05-11T15:52:45,774 copying lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:52:45,777 copying lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:52:45,779 copying lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:52:45,782 copying lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:52:45,784 copying lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yaml -> build/lib/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:52:45,787 copying lm_eval/tasks/afrobench/xlsum/README.md -> build/lib/lm_eval/tasks/afrobench/xlsum 2026-05-11T15:52:45,790 copying lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:52:45,794 copying lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:52:45,797 copying lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum -> build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:52:45,800 copying lm_eval/tasks/afrobench/uhura-arc-easy/README.md -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:52:45,803 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:52:45,806 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:52:45,809 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:52:45,813 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:52:45,816 copying lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy -> build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:52:45,820 copying lm_eval/tasks/afrobench/masakhaner/README.md -> build/lib/lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:52:45,823 copying lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:52:45,826 copying lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:52:45,829 copying lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:52:45,832 copying lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:52:45,835 copying lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner -> build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:52:45,839 copying lm_eval/tasks/afrobench/afriqa/README.md -> build/lib/lm_eval/tasks/afrobench/afriqa 2026-05-11T15:52:45,842 copying lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:52:45,845 copying lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:52:45,848 copying lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:52:45,851 copying lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:52:45,854 copying lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa -> build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:52:45,857 creating build/lib/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T15:52:45,858 copying lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench_lite.sh -> build/lib/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T15:52:45,861 copying lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench.sh -> build/lib/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T15:52:45,864 copying lm_eval/tasks/afrobench/mafand/README.md -> build/lib/lm_eval/tasks/afrobench/mafand 2026-05-11T15:52:45,867 copying lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:52:45,870 copying lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:52:45,874 copying lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:52:45,877 copying lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:52:45,880 copying lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:52:45,884 copying lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand -> build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:52:45,887 copying lm_eval/tasks/afrobench/openai_mmlu/README.md -> build/lib/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:52:45,890 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:52:45,893 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:52:45,895 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:52:45,897 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:52:45,900 copying lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu -> build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:52:45,902 copying lm_eval/tasks/openai-mmmlu/languages.json -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T15:52:45,904 copying lm_eval/tasks/openai-mmmlu/subjects.json -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T15:52:45,907 copying lm_eval/tasks/openai-mmmlu/README.md -> build/lib/lm_eval/tasks/openai-mmmlu 2026-05-11T15:52:45,910 copying lm_eval/tasks/openai-mmmlu/default/_default_template_yaml -> build/lib/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:52:45,913 copying lm_eval/tasks/mts_dialog/README.md -> build/lib/lm_eval/tasks/mts_dialog 2026-05-11T15:52:45,917 copying lm_eval/tasks/meddialog/README.md -> build/lib/lm_eval/tasks/meddialog 2026-05-11T15:52:45,920 copying lm_eval/tasks/cabbq/_cabbq_common_yaml -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:45,923 copying lm_eval/tasks/cabbq/README.md -> build/lib/lm_eval/tasks/cabbq 2026-05-11T15:52:45,926 copying lm_eval/tasks/arithmetic/README.md -> build/lib/lm_eval/tasks/arithmetic 2026-05-11T15:52:45,929 copying lm_eval/tasks/babilong/_babilong_common_yaml -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:45,932 copying lm_eval/tasks/babilong/README.md -> build/lib/lm_eval/tasks/babilong 2026-05-11T15:52:45,936 copying lm_eval/tasks/careqa/README.md -> build/lib/lm_eval/tasks/careqa 2026-05-11T15:52:45,939 copying lm_eval/tasks/mmlu-redux-spanish/README.md -> build/lib/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T15:52:45,942 copying lm_eval/tasks/mmlu-redux-spanish/generative/_default_template_spanish_yaml -> build/lib/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:52:45,946 copying lm_eval/tasks/metabench/README.md -> build/lib/lm_eval/tasks/metabench 2026-05-11T15:52:45,949 copying lm_eval/tasks/llama3/README.md -> build/lib/lm_eval/tasks/llama3 2026-05-11T15:52:45,952 copying lm_eval/tasks/llama3/instruct/mmlu_hi/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:52:45,955 copying lm_eval/tasks/llama3/instruct/mmlu_fr/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:52:45,959 copying lm_eval/tasks/llama3/instruct/mmlu_it/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:52:45,962 copying lm_eval/tasks/llama3/instruct/mmlu_th/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:52:45,965 copying lm_eval/tasks/llama3/instruct/mmlu_de/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:52:45,968 copying lm_eval/tasks/llama3/instruct/mmlu/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:52:45,970 copying lm_eval/tasks/llama3/instruct/mmlu_es/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:52:45,973 copying lm_eval/tasks/llama3/instruct/mmlu_pro/_default_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:52:45,976 copying lm_eval/tasks/llama3/instruct/mmlu_pt/_continuation_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:52:45,979 copying lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama_template_yaml -> build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:52:45,982 copying lm_eval/tasks/copal_id/README.md -> build/lib/lm_eval/tasks/copal_id 2026-05-11T15:52:54,652 installing to build/bdist.linux-armv7l/wheel 2026-05-11T15:52:54,652 running install 2026-05-11T15:52:54,676 running install_lib 2026-05-11T15:52:54,682 creating build/bdist.linux-armv7l/wheel 2026-05-11T15:52:54,685 creating build/bdist.linux-armv7l/wheel/lm_eval 2026-05-11T15:52:54,686 copying build/lib/lm_eval/__main__.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:52:54,688 creating build/bdist.linux-armv7l/wheel/lm_eval/models 2026-05-11T15:52:54,689 copying build/lib/lm_eval/models/mamba_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,692 copying build/lib/lm_eval/models/hf_steered.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,694 copying build/lib/lm_eval/models/vllm_vlms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,696 copying build/lib/lm_eval/models/nemo_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,699 copying build/lib/lm_eval/models/dummy.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,701 copying build/lib/lm_eval/models/gguf.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,703 copying build/lib/lm_eval/models/winml.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,706 copying build/lib/lm_eval/models/optimum_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,707 copying build/lib/lm_eval/models/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,709 copying build/lib/lm_eval/models/utils_hf.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,711 copying build/lib/lm_eval/models/vllm_causallms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,714 copying build/lib/lm_eval/models/optimum_ipex.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,716 copying build/lib/lm_eval/models/api_models.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,719 copying build/lib/lm_eval/models/ibm_watsonx_ai.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,721 copying build/lib/lm_eval/models/hf_audiolm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,723 copying build/lib/lm_eval/models/huggingface.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,727 copying build/lib/lm_eval/models/openai_completions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,730 copying build/lib/lm_eval/models/megatron_lm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,733 copying build/lib/lm_eval/models/anthropic_llms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,735 copying build/lib/lm_eval/models/textsynth.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,737 copying build/lib/lm_eval/models/sglang_generate_API.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,739 copying build/lib/lm_eval/models/hf_vlms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,742 copying build/lib/lm_eval/models/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,745 copying build/lib/lm_eval/models/mistral3.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,747 copying build/lib/lm_eval/models/sglang_causallms.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,750 copying build/lib/lm_eval/models/neuron_optimum.py -> build/bdist.linux-armv7l/wheel/./lm_eval/models 2026-05-11T15:52:54,757 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks 2026-05-11T15:52:54,759 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noticia 2026-05-11T15:52:54,760 copying build/lib/lm_eval/tasks/noticia/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noticia 2026-05-11T15:52:54,762 copying build/lib/lm_eval/tasks/noticia/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noticia 2026-05-11T15:52:54,764 copying build/lib/lm_eval/tasks/noticia/noticia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noticia 2026-05-11T15:52:54,766 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arab_culture 2026-05-11T15:52:54,767 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_algeria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,769 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_ksa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,771 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_tunisia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,773 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,774 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_uae.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,776 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_libya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,778 copying build/lib/lm_eval/tasks/arab_culture/prompts.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,780 copying build/lib/lm_eval/tasks/arab_culture/utils_mcq.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,782 copying build/lib/lm_eval/tasks/arab_culture/_default_arab_culture_mcq_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,783 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,785 copying build/lib/lm_eval/tasks/arab_culture/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,787 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,789 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,791 copying build/lib/lm_eval/tasks/arab_culture/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,793 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,794 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_gulf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,796 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_north_africa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,798 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_sudan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,800 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_morocco.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,801 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_nile_valley.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,803 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,805 copying build/lib/lm_eval/tasks/arab_culture/_arab_culture_levant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,807 copying build/lib/lm_eval/tasks/arab_culture/arab_culture_yemen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture 2026-05-11T15:52:54,809 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus 2026-05-11T15:52:54,810 copying build/lib/lm_eval/tasks/tmmluplus/subject.tsv -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus 2026-05-11T15:52:54,812 copying build/lib/lm_eval/tasks/tmmluplus/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus 2026-05-11T15:52:54,815 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,816 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,818 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,819 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,821 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,823 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,825 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,826 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,828 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,830 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,831 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,833 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,835 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,837 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,838 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,840 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,842 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,844 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,846 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,847 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,849 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,851 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,853 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,854 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,856 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,857 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,859 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,861 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,863 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,865 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,866 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,869 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,870 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,872 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_STEM.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,874 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,876 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,878 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,880 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,881 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,883 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,885 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,887 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,889 copying build/lib/lm_eval/tasks/tmmluplus/default/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,891 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,893 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,894 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,896 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,898 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,900 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,901 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,903 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,905 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,906 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,908 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,910 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,911 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,913 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,915 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,917 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,918 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,920 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,922 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,923 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,925 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,927 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,929 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,930 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,932 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,934 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,935 copying build/lib/lm_eval/tasks/tmmluplus/default/_tmmluplus_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,937 copying build/lib/lm_eval/tasks/tmmluplus/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,939 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,941 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,943 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,944 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,946 copying build/lib/lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmmluplus/default 2026-05-11T15:52:54,948 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mimic_repsum 2026-05-11T15:52:54,949 copying build/lib/lm_eval/tasks/mimic_repsum/mimic_repsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T15:52:54,951 copying build/lib/lm_eval/tasks/mimic_repsum/mimic_repsum_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T15:52:54,953 copying build/lib/lm_eval/tasks/mimic_repsum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T15:52:54,955 copying build/lib/lm_eval/tasks/mimic_repsum/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T15:52:54,957 copying build/lib/lm_eval/tasks/mimic_repsum/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mimic_repsum 2026-05-11T15:52:54,959 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cnn_dailymail 2026-05-11T15:52:54,960 copying build/lib/lm_eval/tasks/cnn_dailymail/cnn_dailymail.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cnn_dailymail 2026-05-11T15:52:54,962 copying build/lib/lm_eval/tasks/cnn_dailymail/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cnn_dailymail 2026-05-11T15:52:54,964 copying build/lib/lm_eval/tasks/cnn_dailymail/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cnn_dailymail 2026-05-11T15:52:54,967 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice 2026-05-11T15:52:54,967 copying build/lib/lm_eval/tasks/aradice/aradice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice 2026-05-11T15:52:54,970 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,971 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,973 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,975 copying build/lib/lm_eval/tasks/aradice/winogrande/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,976 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,978 copying build/lib/lm_eval/tasks/aradice/winogrande/winogrande_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,980 copying build/lib/lm_eval/tasks/aradice/winogrande/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/winogrande 2026-05-11T15:52:54,982 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/ArabicMMLU 2026-05-11T15:52:54,984 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,985 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,987 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,989 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,990 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,992 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,994 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,996 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,997 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:54,999 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,001 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,003 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,004 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,006 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,008 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,010 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,012 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,014 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,016 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,019 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,021 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,023 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,025 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,026 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,028 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,030 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,032 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,034 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,036 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,038 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,040 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,042 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,043 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,045 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,047 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,049 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,052 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,054 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,056 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,058 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,060 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,062 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,064 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,066 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,067 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/LEV 2026-05-11T15:52:55,071 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,072 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,074 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,075 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,077 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,079 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,081 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,083 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,085 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,087 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,089 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,091 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,093 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,095 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,097 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,099 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,101 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,103 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,105 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,107 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,108 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,110 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_other_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,112 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,114 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,116 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,118 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,120 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,122 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,124 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,126 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_prof_humanities_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,128 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,130 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,132 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,134 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,136 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,138 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,140 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,142 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,144 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,146 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,149 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,151 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,153 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_driving-test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,155 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,157 copying build/lib/lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/ArabicMMLU/EGY 2026-05-11T15:52:55,159 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,160 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,162 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,164 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,166 copying build/lib/lm_eval/tasks/aradice/openbookqa/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,168 copying build/lib/lm_eval/tasks/aradice/openbookqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,170 copying build/lib/lm_eval/tasks/aradice/openbookqa/openbookqa_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/openbookqa 2026-05-11T15:52:55,173 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,174 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/qatar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,176 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,178 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,180 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,182 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,184 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,186 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,188 copying build/lib/lm_eval/tasks/aradice/cultural-benchmark/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/cultural-benchmark 2026-05-11T15:52:55,190 copying build/lib/lm_eval/tasks/aradice/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice 2026-05-11T15:52:55,193 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq 2026-05-11T15:52:55,194 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:55,196 copying build/lib/lm_eval/tasks/aradice/boolq/MSA/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:55,198 copying build/lib/lm_eval/tasks/aradice/boolq/MSA/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:55,199 copying build/lib/lm_eval/tasks/aradice/boolq/MSA/boolq_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/MSA 2026-05-11T15:52:55,202 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:55,203 copying build/lib/lm_eval/tasks/aradice/boolq/LEV/boolq_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:55,206 copying build/lib/lm_eval/tasks/aradice/boolq/LEV/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:55,207 copying build/lib/lm_eval/tasks/aradice/boolq/LEV/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/LEV 2026-05-11T15:52:55,210 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:55,211 copying build/lib/lm_eval/tasks/aradice/boolq/EGY/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:55,213 copying build/lib/lm_eval/tasks/aradice/boolq/EGY/boolq_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:55,215 copying build/lib/lm_eval/tasks/aradice/boolq/EGY/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/EGY 2026-05-11T15:52:55,218 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:55,219 copying build/lib/lm_eval/tasks/aradice/boolq/ENG/boolq_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:55,222 copying build/lib/lm_eval/tasks/aradice/boolq/ENG/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:55,224 copying build/lib/lm_eval/tasks/aradice/boolq/ENG/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/boolq/ENG 2026-05-11T15:52:55,226 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/piqa 2026-05-11T15:52:55,228 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T15:52:55,230 copying build/lib/lm_eval/tasks/aradice/piqa/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T15:52:55,232 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T15:52:55,235 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T15:52:55,237 copying build/lib/lm_eval/tasks/aradice/piqa/piqa_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/piqa 2026-05-11T15:52:55,239 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:55,240 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:55,243 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:55,245 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:55,247 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_lev.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:55,249 copying build/lib/lm_eval/tasks/aradice/truthfulqa_mcq/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aradice/truthfulqa_mcq 2026-05-11T15:52:55,251 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux 2026-05-11T15:52:55,254 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,255 copying build/lib/lm_eval/tasks/mmlu-redux/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,258 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,260 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,262 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,264 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,266 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,268 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,270 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,272 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,274 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,276 copying build/lib/lm_eval/tasks/mmlu-redux/generative/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,278 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,280 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,282 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,283 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,285 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,287 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,289 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,291 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,292 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,294 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,296 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,297 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,299 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,301 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,302 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,304 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,306 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,307 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,309 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,311 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,313 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,315 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,316 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,318 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,320 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,321 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,323 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,325 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,327 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,329 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,331 copying build/lib/lm_eval/tasks/mmlu-redux/generative/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,332 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,334 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,336 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,337 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,339 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,341 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,343 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,345 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,346 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,348 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,350 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,352 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,354 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,355 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,357 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,359 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,361 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,362 copying build/lib/lm_eval/tasks/mmlu-redux/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux/generative 2026-05-11T15:52:55,365 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mc_taco 2026-05-11T15:52:55,366 copying build/lib/lm_eval/tasks/mc_taco/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mc_taco 2026-05-11T15:52:55,368 copying build/lib/lm_eval/tasks/mc_taco/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mc_taco 2026-05-11T15:52:55,370 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/humaneval 2026-05-11T15:52:55,371 copying build/lib/lm_eval/tasks/humaneval/humaneval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,373 copying build/lib/lm_eval/tasks/humaneval/humaneval_64_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,375 copying build/lib/lm_eval/tasks/humaneval/humaneval_64.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,377 copying build/lib/lm_eval/tasks/humaneval/humaneval_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,378 copying build/lib/lm_eval/tasks/humaneval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,380 copying build/lib/lm_eval/tasks/humaneval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,382 copying build/lib/lm_eval/tasks/humaneval/humaneval_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval 2026-05-11T15:52:55,386 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/multiblimp 2026-05-11T15:52:55,387 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,389 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kxh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,390 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,392 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kpv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,393 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_uzb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,395 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,397 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_quc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,399 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hye.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,400 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_got.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,402 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_slv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,404 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_orv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,406 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,408 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_eus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,410 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_rus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,412 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_vep.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,413 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_nds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,415 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,417 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_tur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,419 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,421 copying build/lib/lm_eval/tasks/multiblimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,423 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_urd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,425 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_glg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,427 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ota.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,429 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_krl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,431 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_abk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,432 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,434 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_urb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,436 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hun.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,438 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_xpg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,440 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_mdf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,441 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_wbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,443 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_yrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,445 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_nhi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,447 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,448 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lav.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,450 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_frm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,452 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_olo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,454 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kmr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,456 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ell.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,458 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_cym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,460 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,462 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_dan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,463 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bel.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,465 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,467 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ttc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,469 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fas.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,471 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,473 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,475 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_nld.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,477 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_aqz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,479 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_swe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,481 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,483 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_pol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,485 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_heb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,486 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ron.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,489 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_xnr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,491 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_deu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,492 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,494 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_tpn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,496 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,498 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_san.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,500 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_xcl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,502 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,504 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_spa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,506 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bre.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,507 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,509 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_egy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,511 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,513 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_koi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,515 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,517 copying build/lib/lm_eval/tasks/multiblimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,519 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_guj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,521 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_apu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,523 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_kir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,525 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,527 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_mar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,528 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ces.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,530 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ben.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,532 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_mkd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,533 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_est.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,535 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_tam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,537 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_lij.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,539 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_chu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,540 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_bua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,542 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ita.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,544 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_uig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,546 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sqi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,547 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_myv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,549 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_gle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,551 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_isl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,553 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_sah.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,555 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_azz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,557 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hyw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,559 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_arb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,560 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_grc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,562 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_hsb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,564 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,566 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_ukr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,567 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_slk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,569 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_cat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,571 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_aln.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,573 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,575 copying build/lib/lm_eval/tasks/multiblimp/multiblimp_gla.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/multiblimp 2026-05-11T15:52:55,577 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/commonsense_qa 2026-05-11T15:52:55,578 copying build/lib/lm_eval/tasks/commonsense_qa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/commonsense_qa 2026-05-11T15:52:55,580 copying build/lib/lm_eval/tasks/commonsense_qa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/commonsense_qa 2026-05-11T15:52:55,582 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc 2026-05-11T15:52:55,584 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,585 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/_var5shots_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,587 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Material.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,589 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,590 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,592 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,594 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/_sciknoweval_mcqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,596 copying build/lib/lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/sciknoweval_mcqa 2026-05-11T15:52:55,598 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,600 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,601 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,603 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,605 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,607 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,608 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,610 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,612 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,614 copying build/lib/lm_eval/tasks/e2lmc/noor/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,616 copying build/lib/lm_eval/tasks/e2lmc/noor/_noor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,618 copying build/lib/lm_eval/tasks/e2lmc/noor/_noor_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,619 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,621 copying build/lib/lm_eval/tasks/e2lmc/noor/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,623 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,625 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,627 copying build/lib/lm_eval/tasks/e2lmc/noor/noor_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,629 copying build/lib/lm_eval/tasks/e2lmc/noor/_noor_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/noor 2026-05-11T15:52:55,631 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:55,632 copying build/lib/lm_eval/tasks/e2lmc/mmlu_early_training/custom_metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:55,634 copying build/lib/lm_eval/tasks/e2lmc/mmlu_early_training/mmlu_early_training.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:55,635 copying build/lib/lm_eval/tasks/e2lmc/mmlu_early_training/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/e2lmc/mmlu_early_training 2026-05-11T15:52:55,638 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paloma 2026-05-11T15:52:55,639 copying build/lib/lm_eval/tasks/paloma/paloma_wikitext_103.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,640 copying build/lib/lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,642 copying build/lib/lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,644 copying build/lib/lm_eval/tasks/paloma/paloma_redpajama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,646 copying build/lib/lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,647 copying build/lib/lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,649 copying build/lib/lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,651 copying build/lib/lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,652 copying build/lib/lm_eval/tasks/paloma/paloma_gab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,654 copying build/lib/lm_eval/tasks/paloma/paloma_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,655 copying build/lib/lm_eval/tasks/paloma/_paloma_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,657 copying build/lib/lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,659 copying build/lib/lm_eval/tasks/paloma/paloma_c4_100_domains.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,660 copying build/lib/lm_eval/tasks/paloma/paloma_mc4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,662 copying build/lib/lm_eval/tasks/paloma/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,664 copying build/lib/lm_eval/tasks/paloma/paloma_c4_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,666 copying build/lib/lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,667 copying build/lib/lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,669 copying build/lib/lm_eval/tasks/paloma/paloma_ptb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paloma 2026-05-11T15:52:55,671 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/c4 2026-05-11T15:52:55,673 copying build/lib/lm_eval/tasks/c4/preprocess_c4.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/c4 2026-05-11T15:52:55,675 copying build/lib/lm_eval/tasks/c4/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/c4 2026-05-11T15:52:55,676 copying build/lib/lm_eval/tasks/c4/c4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/c4 2026-05-11T15:52:55,679 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox 2026-05-11T15:52:55,681 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,682 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,683 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,685 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,687 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,689 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,690 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,692 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,694 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_lite_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,695 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,697 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,700 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,701 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,703 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,705 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,707 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,708 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,711 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,713 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,715 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,717 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,719 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,721 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,723 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,724 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,726 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,728 copying build/lib/lm_eval/tasks/mmlu_prox/mr/_mr_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,730 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,731 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,733 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,735 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,737 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,739 copying build/lib/lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,741 copying build/lib/lm_eval/tasks/mmlu_prox/mr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/mr 2026-05-11T15:52:55,743 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,744 copying build/lib/lm_eval/tasks/mmlu_prox/de/_mmlu_prox_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,746 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,748 copying build/lib/lm_eval/tasks/mmlu_prox/de/_de_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,750 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,752 copying build/lib/lm_eval/tasks/mmlu_prox/de/_de_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,754 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,756 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,758 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,760 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,762 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,764 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,766 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,768 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,770 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,772 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,774 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,776 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,778 copying build/lib/lm_eval/tasks/mmlu_prox/de/_mmlu_prox_lite_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,780 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,782 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,784 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,785 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,787 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,789 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,791 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,794 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,795 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,797 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,799 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,801 copying build/lib/lm_eval/tasks/mmlu_prox/de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,803 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,805 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,808 copying build/lib/lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/de 2026-05-11T15:52:55,810 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,812 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,814 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,816 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,818 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_lite_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,820 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,822 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,824 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,826 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,827 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,829 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,831 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,833 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,835 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,837 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,839 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,841 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,842 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,844 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,846 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,848 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,850 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_yo_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,852 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,854 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,856 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,858 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,860 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,861 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,863 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,865 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,867 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,869 copying build/lib/lm_eval/tasks/mmlu_prox/yo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,872 copying build/lib/lm_eval/tasks/mmlu_prox/yo/_yo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,873 copying build/lib/lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/yo 2026-05-11T15:52:55,875 copying build/lib/lm_eval/tasks/mmlu_prox/lang_libs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T15:52:55,879 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,880 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_ru_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,882 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,884 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,886 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,888 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,890 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,891 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,893 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,895 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,897 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,899 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,901 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,903 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,905 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,907 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,909 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,910 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,912 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,914 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,916 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,918 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,920 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,922 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,924 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,926 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,928 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_lite_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,930 copying build/lib/lm_eval/tasks/mmlu_prox/ru/_ru_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,932 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,934 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,936 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,937 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,939 copying build/lib/lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,941 copying build/lib/lm_eval/tasks/mmlu_prox/ru/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ru 2026-05-11T15:52:55,945 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,946 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,948 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,950 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_zh_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,952 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,954 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_zh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,956 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,958 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,959 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,961 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,963 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,965 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,967 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,969 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,970 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,972 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,974 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,976 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,977 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,979 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,981 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,983 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,984 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,986 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,988 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,990 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,992 copying build/lib/lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_lite_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,994 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,996 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:55,997 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:56,000 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:56,002 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:56,004 copying build/lib/lm_eval/tasks/mmlu_prox/zh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:56,006 copying build/lib/lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zh 2026-05-11T15:52:56,009 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,010 copying build/lib/lm_eval/tasks/mmlu_prox/af/_mmlu_prox_lite_af.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,012 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,014 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,016 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,017 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,020 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,021 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,023 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,025 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,027 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,029 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,030 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,032 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,034 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,036 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,038 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,040 copying build/lib/lm_eval/tasks/mmlu_prox/af/_af_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,042 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,044 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,046 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,048 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,050 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,052 copying build/lib/lm_eval/tasks/mmlu_prox/af/_mmlu_prox_af.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,054 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,056 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,058 copying build/lib/lm_eval/tasks/mmlu_prox/af/_af_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,060 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,062 copying build/lib/lm_eval/tasks/mmlu_prox/af/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,065 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,067 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,069 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,071 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,073 copying build/lib/lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/af 2026-05-11T15:52:56,076 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,077 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,079 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,081 copying build/lib/lm_eval/tasks/mmlu_prox/th/_mmlu_prox_lite_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,082 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,084 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,086 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,088 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,090 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,092 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,094 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,096 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,097 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,100 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,101 copying build/lib/lm_eval/tasks/mmlu_prox/th/_th_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,103 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,105 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,107 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,109 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,111 copying build/lib/lm_eval/tasks/mmlu_prox/th/_th_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,113 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,115 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,117 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,119 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,121 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,123 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,125 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,127 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,129 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,131 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,133 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,135 copying build/lib/lm_eval/tasks/mmlu_prox/th/_mmlu_prox_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,137 copying build/lib/lm_eval/tasks/mmlu_prox/th/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,139 copying build/lib/lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/th 2026-05-11T15:52:56,142 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,143 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,145 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,147 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,149 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,151 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,153 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,154 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,156 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,158 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,160 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,162 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,164 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,166 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,168 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,170 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,172 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,174 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_lite_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,175 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,177 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,179 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,181 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,183 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,184 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,186 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,188 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,190 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_zu_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,192 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,194 copying build/lib/lm_eval/tasks/mmlu_prox/zu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,196 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,198 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,200 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,201 copying build/lib/lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,203 copying build/lib/lm_eval/tasks/mmlu_prox/zu/_zu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/zu 2026-05-11T15:52:56,206 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,207 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,210 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,211 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,213 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,215 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,217 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,219 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,221 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,223 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_lite_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,225 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,227 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,229 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,231 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,233 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,235 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,237 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,239 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,241 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,243 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,245 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,246 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,248 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,250 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,252 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_ne_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,254 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,256 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,258 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,260 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,262 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,264 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,266 copying build/lib/lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,268 copying build/lib/lm_eval/tasks/mmlu_prox/ne/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,270 copying build/lib/lm_eval/tasks/mmlu_prox/ne/_ne_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ne 2026-05-11T15:52:56,273 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,274 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,276 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,277 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_vi_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,279 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,281 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,283 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_vi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,285 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,287 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,289 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,291 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,292 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,294 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,296 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,298 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_lite_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,300 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,302 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,304 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,306 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,308 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,310 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,312 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,314 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,316 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,318 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,320 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,321 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,323 copying build/lib/lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,325 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,327 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,328 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,330 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,331 copying build/lib/lm_eval/tasks/mmlu_prox/vi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,333 copying build/lib/lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/vi 2026-05-11T15:52:56,336 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,337 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,339 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,341 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_ko_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,342 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,344 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,346 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,348 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,349 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,351 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,353 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,355 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,357 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,359 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_ko_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,361 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,363 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,364 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,366 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,368 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,370 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,371 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,373 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,375 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,377 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,379 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,381 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_lite_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,383 copying build/lib/lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,385 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,387 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,389 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,391 copying build/lib/lm_eval/tasks/mmlu_prox/ko/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,393 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,394 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,396 copying build/lib/lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ko 2026-05-11T15:52:56,399 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,400 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,402 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_bn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,404 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_lite_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,406 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,407 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,409 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,411 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,413 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,415 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,417 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,418 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,420 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,422 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,424 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,426 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,428 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,430 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,432 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,434 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,436 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,438 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,439 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,441 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,443 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,445 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,447 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,449 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,451 copying build/lib/lm_eval/tasks/mmlu_prox/bn/_bn_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,453 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,455 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,457 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,459 copying build/lib/lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,461 copying build/lib/lm_eval/tasks/mmlu_prox/bn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/bn 2026-05-11T15:52:56,463 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/template 2026-05-11T15:52:56,464 copying build/lib/lm_eval/tasks/mmlu_prox/template/_lang_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/template 2026-05-11T15:52:56,466 copying build/lib/lm_eval/tasks/mmlu_prox/template/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/template 2026-05-11T15:52:56,469 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,470 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,473 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,474 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,476 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,478 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,480 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,482 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,484 copying build/lib/lm_eval/tasks/mmlu_prox/en/_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,486 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,488 copying build/lib/lm_eval/tasks/mmlu_prox/en/_mmlu_prox_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,490 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,491 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,493 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,495 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,498 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,500 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,502 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,503 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,505 copying build/lib/lm_eval/tasks/mmlu_prox/en/_mmlu_prox_lite_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,507 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,509 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,511 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,513 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,515 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,516 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,518 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,520 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,522 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,524 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,526 copying build/lib/lm_eval/tasks/mmlu_prox/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,528 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,530 copying build/lib/lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,532 copying build/lib/lm_eval/tasks/mmlu_prox/en/_en_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/en 2026-05-11T15:52:56,535 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,537 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,539 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_uk_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,541 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_uk_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,542 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,544 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,546 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,548 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,550 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,552 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,554 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,556 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,558 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,560 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,562 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,564 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,565 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,567 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,569 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,571 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,573 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,575 copying build/lib/lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_lite_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,577 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,579 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,581 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,583 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,585 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,586 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,588 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,590 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,592 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,594 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,596 copying build/lib/lm_eval/tasks/mmlu_prox/uk/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,598 copying build/lib/lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/uk 2026-05-11T15:52:56,601 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,602 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,604 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,606 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,608 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,610 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,612 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,614 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,616 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,618 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,620 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,621 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,623 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_sw_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,625 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,627 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,629 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,631 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,633 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,634 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,636 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,638 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,640 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,642 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,644 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,646 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,648 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_lite_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,650 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,652 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,654 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,655 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,657 copying build/lib/lm_eval/tasks/mmlu_prox/sw/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,659 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,661 copying build/lib/lm_eval/tasks/mmlu_prox/sw/_sw_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,663 copying build/lib/lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sw 2026-05-11T15:52:56,666 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,667 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,669 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,671 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,673 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,675 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,677 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,679 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,681 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,683 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,684 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,686 copying build/lib/lm_eval/tasks/mmlu_prox/es/_es_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,688 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,690 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,692 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,694 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,696 copying build/lib/lm_eval/tasks/mmlu_prox/es/_es_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,697 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,699 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,701 copying build/lib/lm_eval/tasks/mmlu_prox/es/_mmlu_prox_lite_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,703 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,705 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,707 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,709 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,711 copying build/lib/lm_eval/tasks/mmlu_prox/es/_mmlu_prox_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,713 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,714 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,716 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,718 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,720 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,722 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,724 copying build/lib/lm_eval/tasks/mmlu_prox/es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,726 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,728 copying build/lib/lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/es 2026-05-11T15:52:56,731 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,732 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,734 copying build/lib/lm_eval/tasks/mmlu_prox/te/_mmlu_prox_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,736 copying build/lib/lm_eval/tasks/mmlu_prox/te/_te_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,738 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,740 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,742 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,743 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,745 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,747 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,749 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,751 copying build/lib/lm_eval/tasks/mmlu_prox/te/_mmlu_prox_lite_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,753 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,755 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,757 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,759 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,760 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,762 copying build/lib/lm_eval/tasks/mmlu_prox/te/_te_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,764 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,766 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,768 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,770 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,771 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,773 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,775 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,777 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,779 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,781 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,783 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,785 copying build/lib/lm_eval/tasks/mmlu_prox/te/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,787 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,789 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,791 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,793 copying build/lib/lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/te 2026-05-11T15:52:56,796 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,797 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,799 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_ja_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,801 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,803 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,805 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,807 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,808 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_lite_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,810 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,812 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,814 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,816 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,817 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,819 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,821 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,823 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,825 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,826 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,829 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,831 copying build/lib/lm_eval/tasks/mmlu_prox/ja/_ja_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,833 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,834 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,836 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,838 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,840 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,842 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,844 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,845 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,847 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,849 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,851 copying build/lib/lm_eval/tasks/mmlu_prox/ja/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,853 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,855 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,856 copying build/lib/lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ja 2026-05-11T15:52:56,859 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,860 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,862 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,864 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_hi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,866 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,867 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,869 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,871 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,873 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,875 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,877 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,878 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,880 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,883 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,885 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,887 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_lite_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,889 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,891 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,893 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,894 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,896 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,898 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,900 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,902 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,904 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,905 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,907 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,909 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,911 copying build/lib/lm_eval/tasks/mmlu_prox/hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,913 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,915 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,916 copying build/lib/lm_eval/tasks/mmlu_prox/hi/_hi_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,918 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,920 copying build/lib/lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hi 2026-05-11T15:52:56,922 copying build/lib/lm_eval/tasks/mmlu_prox/mmlu_prox_lite_config_generator.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T15:52:56,925 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,926 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,929 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,931 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,932 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,935 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,937 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,939 copying build/lib/lm_eval/tasks/mmlu_prox/it/_it_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,940 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,942 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,944 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,946 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,948 copying build/lib/lm_eval/tasks/mmlu_prox/it/_mmlu_prox_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,950 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,952 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,954 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,956 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,958 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,959 copying build/lib/lm_eval/tasks/mmlu_prox/it/_it_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,961 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,963 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,965 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,967 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,969 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,971 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,972 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,974 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,976 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,978 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,980 copying build/lib/lm_eval/tasks/mmlu_prox/it/_mmlu_prox_lite_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,982 copying build/lib/lm_eval/tasks/mmlu_prox/it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,984 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,986 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,988 copying build/lib/lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/it 2026-05-11T15:52:56,990 copying build/lib/lm_eval/tasks/mmlu_prox/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T15:52:56,993 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:56,994 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:56,997 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:56,999 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,001 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,003 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,005 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,007 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,009 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,011 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,013 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,015 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,017 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,019 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,021 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_sr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,022 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,024 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,026 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,028 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,030 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,032 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,034 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,036 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,038 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,040 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,042 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,044 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,046 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,047 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,049 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,051 copying build/lib/lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,053 copying build/lib/lm_eval/tasks/mmlu_prox/sr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,055 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_lite_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,057 copying build/lib/lm_eval/tasks/mmlu_prox/sr/_sr_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/sr 2026-05-11T15:52:57,060 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,061 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_wo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,063 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,065 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,067 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,069 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,071 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,073 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_wo_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,075 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,076 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,079 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,080 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,082 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,084 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,086 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,088 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,090 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,092 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,094 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,096 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,098 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,100 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,102 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,104 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,106 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,108 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,110 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,112 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,113 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,115 copying build/lib/lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_lite_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,117 copying build/lib/lm_eval/tasks/mmlu_prox/wo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,119 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,121 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,123 copying build/lib/lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/wo 2026-05-11T15:52:57,125 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,127 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_lite_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,129 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,131 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,133 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,134 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,136 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,138 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,140 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,142 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,144 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,145 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,147 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,149 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,151 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,153 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,155 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_fr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,157 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,158 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,160 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,162 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,163 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,165 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,167 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,169 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_fr_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,171 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,173 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,175 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,176 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,178 copying build/lib/lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,180 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,182 copying build/lib/lm_eval/tasks/mmlu_prox/fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,184 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,186 copying build/lib/lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/fr 2026-05-11T15:52:57,189 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,190 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,192 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,194 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,195 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_lite_cs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,197 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_cs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,199 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,201 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,203 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,205 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,206 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,208 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_cs_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,210 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,212 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,214 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,216 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,218 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,220 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,222 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,224 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,225 copying build/lib/lm_eval/tasks/mmlu_prox/cs/_cs_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,227 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,229 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,231 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,233 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,235 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,237 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,240 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,242 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,243 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,245 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,247 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,249 copying build/lib/lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,251 copying build/lib/lm_eval/tasks/mmlu_prox/cs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/cs 2026-05-11T15:52:57,255 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,256 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,258 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,260 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,262 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,264 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,266 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,268 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,270 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,272 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,273 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,275 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,277 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,279 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,281 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,283 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,284 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_pt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,286 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,288 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,290 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,292 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,294 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,296 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,298 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,299 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,301 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,303 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,305 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_lite_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,306 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,308 copying build/lib/lm_eval/tasks/mmlu_prox/pt/_pt_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,310 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,312 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,314 copying build/lib/lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,316 copying build/lib/lm_eval/tasks/mmlu_prox/pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/pt 2026-05-11T15:52:57,319 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,320 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,322 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,324 copying build/lib/lm_eval/tasks/mmlu_prox/id/_id_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,326 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,327 copying build/lib/lm_eval/tasks/mmlu_prox/id/_mmlu_prox_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,329 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,331 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,333 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,335 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,337 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,339 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,341 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,343 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,345 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,346 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,348 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,350 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,352 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,354 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,355 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,357 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,359 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,361 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,363 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,364 copying build/lib/lm_eval/tasks/mmlu_prox/id/_id_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,366 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,368 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,370 copying build/lib/lm_eval/tasks/mmlu_prox/id/_mmlu_prox_lite_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,372 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,374 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,376 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,377 copying build/lib/lm_eval/tasks/mmlu_prox/id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,379 copying build/lib/lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/id 2026-05-11T15:52:57,382 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,383 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,385 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_ar_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,387 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,389 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_ar_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,391 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,393 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,394 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,396 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,398 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,400 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,401 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,403 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,405 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,407 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,408 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,410 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,412 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,414 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,416 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,418 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,420 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,422 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,424 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,425 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,427 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,429 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,431 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,433 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,435 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,437 copying build/lib/lm_eval/tasks/mmlu_prox/ar/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,439 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,440 copying build/lib/lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_lite_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,442 copying build/lib/lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ar 2026-05-11T15:52:57,443 copying build/lib/lm_eval/tasks/mmlu_prox/mmlu_prox_config_generator.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox 2026-05-11T15:52:57,447 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,448 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,450 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,452 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,453 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,455 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,457 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,459 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_ur_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,461 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,463 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_lite_ur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,466 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,468 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,470 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,472 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,475 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_ur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,477 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,479 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,482 copying build/lib/lm_eval/tasks/mmlu_prox/ur/_ur_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,484 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,486 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,489 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,491 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,493 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,495 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,498 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,500 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,502 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,504 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,506 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,509 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,511 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,513 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,516 copying build/lib/lm_eval/tasks/mmlu_prox/ur/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,518 copying build/lib/lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/ur 2026-05-11T15:52:57,521 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,523 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,525 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,527 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,530 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,532 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,534 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,536 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_lite_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,539 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,541 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,543 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,546 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,548 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,550 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,552 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,555 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,557 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,559 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,561 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,563 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,565 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,566 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,568 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,570 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,573 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_hu_lite_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,574 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,576 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,578 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,580 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,582 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,584 copying build/lib/lm_eval/tasks/mmlu_prox/hu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,586 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,588 copying build/lib/lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,590 copying build/lib/lm_eval/tasks/mmlu_prox/hu/_hu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_prox/hu 2026-05-11T15:52:57,592 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa 2026-05-11T15:52:57,594 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/piqa_ar 2026-05-11T15:52:57,595 copying build/lib/lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/piqa_ar 2026-05-11T15:52:57,597 copying build/lib/lm_eval/tasks/alghafa/piqa_ar/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/piqa_ar 2026-05-11T15:52:57,600 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/alghafa/copa_ar 2026-05-11T15:52:57,601 copying build/lib/lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/copa_ar 2026-05-11T15:52:57,603 copying build/lib/lm_eval/tasks/alghafa/copa_ar/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/alghafa/copa_ar 2026-05-11T15:52:57,605 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/graphwalks 2026-05-11T15:52:57,606 copying build/lib/lm_eval/tasks/graphwalks/graphwalks_128k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T15:52:57,608 copying build/lib/lm_eval/tasks/graphwalks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T15:52:57,610 copying build/lib/lm_eval/tasks/graphwalks/graphwalks_1M.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T15:52:57,612 copying build/lib/lm_eval/tasks/graphwalks/graphwalks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T15:52:57,614 copying build/lib/lm_eval/tasks/graphwalks/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/graphwalks 2026-05-11T15:52:57,616 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:57,617 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:57,619 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:57,621 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:57,623 copying build/lib/lm_eval/tasks/simple_cooccurrence_bias/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/simple_cooccurrence_bias 2026-05-11T15:52:57,626 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,627 copying build/lib/lm_eval/tasks/catalan_bench/xnli_va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,630 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T15:52:57,631 copying build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va/truthfulqa_va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T15:52:57,633 copying build/lib/lm_eval/tasks/catalan_bench/truthfulqa_va/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/truthfulqa_va 2026-05-11T15:52:57,636 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:57,637 copying build/lib/lm_eval/tasks/catalan_bench/phrases_va/_phrases_va_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:57,639 copying build/lib/lm_eval/tasks/catalan_bench/phrases_va/phrases_va-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:57,641 copying build/lib/lm_eval/tasks/catalan_bench/phrases_va/phrases_ca-va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/phrases_va 2026-05-11T15:52:57,643 copying build/lib/lm_eval/tasks/catalan_bench/openbookqa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,645 copying build/lib/lm_eval/tasks/catalan_bench/teca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,647 copying build/lib/lm_eval/tasks/catalan_bench/catcola.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,649 copying build/lib/lm_eval/tasks/catalan_bench/piqa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,652 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,653 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_pt-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,655 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,657 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,659 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,661 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,663 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_eu-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,664 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_it-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,666 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/create_yamls_flores_ca.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,669 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,671 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,672 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,674 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_es-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,676 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_fr-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,678 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_de-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,680 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,682 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,684 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_en-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,686 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_gl-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,687 copying build/lib/lm_eval/tasks/catalan_bench/flores_ca/flores_ca-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench/flores_ca 2026-05-11T15:52:57,689 copying build/lib/lm_eval/tasks/catalan_bench/parafraseja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,691 copying build/lib/lm_eval/tasks/catalan_bench/mgsm_direct_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,693 copying build/lib/lm_eval/tasks/catalan_bench/paws_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,695 copying build/lib/lm_eval/tasks/catalan_bench/xquad_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,697 copying build/lib/lm_eval/tasks/catalan_bench/arc_ca_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,699 copying build/lib/lm_eval/tasks/catalan_bench/copa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,701 copying build/lib/lm_eval/tasks/catalan_bench/_cabreu_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,703 copying build/lib/lm_eval/tasks/catalan_bench/coqcat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,705 copying build/lib/lm_eval/tasks/catalan_bench/xnli_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,707 copying build/lib/lm_eval/tasks/catalan_bench/_arc_ca_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,709 copying build/lib/lm_eval/tasks/catalan_bench/siqa_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,710 copying build/lib/lm_eval/tasks/catalan_bench/cabreu_abstractive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,712 copying build/lib/lm_eval/tasks/catalan_bench/catalan_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,714 copying build/lib/lm_eval/tasks/catalan_bench/cabreu_extreme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,715 copying build/lib/lm_eval/tasks/catalan_bench/cabreu_extractive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,717 copying build/lib/lm_eval/tasks/catalan_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,719 copying build/lib/lm_eval/tasks/catalan_bench/catalanqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,721 copying build/lib/lm_eval/tasks/catalan_bench/cocoteros_va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,723 copying build/lib/lm_eval/tasks/catalan_bench/arc_ca_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,725 copying build/lib/lm_eval/tasks/catalan_bench/xstorycloze_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,727 copying build/lib/lm_eval/tasks/catalan_bench/wnli_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,729 copying build/lib/lm_eval/tasks/catalan_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/catalan_bench 2026-05-11T15:52:57,732 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bangla 2026-05-11T15:52:57,733 copying build/lib/lm_eval/tasks/bangla/bangla_commonsenseqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T15:52:57,735 copying build/lib/lm_eval/tasks/bangla/bangla_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T15:52:57,737 copying build/lib/lm_eval/tasks/bangla/bangla_openbookqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T15:52:57,739 copying build/lib/lm_eval/tasks/bangla/bangla_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T15:52:57,740 copying build/lib/lm_eval/tasks/bangla/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T15:52:57,742 copying build/lib/lm_eval/tasks/bangla/bangla_boolqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bangla 2026-05-11T15:52:57,745 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogender 2026-05-11T15:52:57,746 copying build/lib/lm_eval/tasks/winogender/winogender_gotcha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,748 copying build/lib/lm_eval/tasks/winogender/winogender_gotcha_female.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,750 copying build/lib/lm_eval/tasks/winogender/winogender_male.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,751 copying build/lib/lm_eval/tasks/winogender/winogender_neutral.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,753 copying build/lib/lm_eval/tasks/winogender/winogender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,755 copying build/lib/lm_eval/tasks/winogender/winogender_female.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,757 copying build/lib/lm_eval/tasks/winogender/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,759 copying build/lib/lm_eval/tasks/winogender/winogender_gotcha_male.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,760 copying build/lib/lm_eval/tasks/winogender/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogender 2026-05-11T15:52:57,763 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/olaph 2026-05-11T15:52:57,764 copying build/lib/lm_eval/tasks/olaph/olaph_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T15:52:57,766 copying build/lib/lm_eval/tasks/olaph/olaph.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T15:52:57,767 copying build/lib/lm_eval/tasks/olaph/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T15:52:57,769 copying build/lib/lm_eval/tasks/olaph/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T15:52:57,771 copying build/lib/lm_eval/tasks/olaph/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/olaph 2026-05-11T15:52:57,773 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:57,775 copying build/lib/lm_eval/tasks/mediqa_qa2019/mediqa_qa2019_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:57,777 copying build/lib/lm_eval/tasks/mediqa_qa2019/mediqa_qa2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:57,779 copying build/lib/lm_eval/tasks/mediqa_qa2019/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:57,781 copying build/lib/lm_eval/tasks/mediqa_qa2019/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:57,783 copying build/lib/lm_eval/tasks/mediqa_qa2019/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mediqa_qa2019 2026-05-11T15:52:57,785 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile_10k 2026-05-11T15:52:57,786 copying build/lib/lm_eval/tasks/pile_10k/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile_10k 2026-05-11T15:52:57,788 copying build/lib/lm_eval/tasks/pile_10k/pile_10k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile_10k 2026-05-11T15:52:57,791 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi 2026-05-11T15:52:57,793 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,794 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,796 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,798 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,800 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,802 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,804 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,806 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,808 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,810 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,811 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,813 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,815 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,816 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,818 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,820 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,822 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,824 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,826 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,827 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,829 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,831 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,833 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,835 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,837 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,839 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,841 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,842 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,844 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,846 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,848 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,850 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,852 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,853 copying build/lib/lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/hellaswag_multilingual 2026-05-11T15:52:57,856 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,857 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,859 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,861 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,863 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,864 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,866 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,868 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,870 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,872 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,873 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,875 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,877 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,879 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,881 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,883 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,885 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,887 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,889 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,891 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,893 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,895 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,897 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,898 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,900 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,902 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,904 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,906 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,907 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,909 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,911 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,913 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,914 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,916 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,918 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,920 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,922 copying build/lib/lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/mmlu_multilingual 2026-05-11T15:52:57,925 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,926 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,928 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,930 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,932 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,934 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,935 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,937 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,939 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,941 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,942 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,944 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,946 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,947 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,949 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,951 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,952 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,954 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,956 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,957 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,959 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,961 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,963 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,965 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,966 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,968 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,970 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,972 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,973 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,975 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,977 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,979 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,981 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,982 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/_arc_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,984 copying build/lib/lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/arc_multilingual 2026-05-11T15:52:57,987 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,988 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,990 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,992 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,993 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,995 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,997 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:57,999 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,001 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,003 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,004 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,006 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,008 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,010 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,011 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,013 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,015 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,017 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,019 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,021 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,023 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,024 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,026 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,028 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,029 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,031 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,033 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,035 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,037 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,039 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,040 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,042 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,044 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,046 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,048 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,050 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,051 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,053 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,055 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,057 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,059 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,060 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,062 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,064 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,066 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,068 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,070 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,071 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,073 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,075 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,076 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,078 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,080 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,082 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,083 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,085 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,087 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,089 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,091 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,092 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,094 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,096 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,098 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,100 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,101 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,103 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,105 copying build/lib/lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/okapi/truthfulqa_multilingual 2026-05-11T15:52:58,108 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_exams 2026-05-11T15:52:58,109 copying build/lib/lm_eval/tasks/eus_exams/configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,111 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,112 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,114 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,116 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,118 copying build/lib/lm_eval/tasks/eus_exams/eus_exams -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,120 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,122 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,123 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,125 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,127 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,129 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,131 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,132 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,134 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,136 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,138 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,140 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,141 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,143 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,145 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,147 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,148 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,150 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,152 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,153 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,155 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,157 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,159 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,160 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,162 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,164 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,166 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,168 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,170 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,171 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,173 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,175 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,177 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,179 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,180 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,182 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,184 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,186 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,188 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,189 copying build/lib/lm_eval/tasks/eus_exams/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,191 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,193 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,195 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,197 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,199 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,200 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,202 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,204 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,206 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,208 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,209 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,211 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,213 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,215 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,217 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,219 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,221 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,222 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,224 copying build/lib/lm_eval/tasks/eus_exams/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,226 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,228 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_eu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,230 copying build/lib/lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_exams 2026-05-11T15:52:58,232 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qasper 2026-05-11T15:52:58,233 copying build/lib/lm_eval/tasks/qasper/freeform.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T15:52:58,236 copying build/lib/lm_eval/tasks/qasper/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T15:52:58,238 copying build/lib/lm_eval/tasks/qasper/bool.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T15:52:58,240 copying build/lib/lm_eval/tasks/qasper/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T15:52:58,242 copying build/lib/lm_eval/tasks/qasper/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qasper 2026-05-11T15:52:58,245 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aexams 2026-05-11T15:52:58,246 copying build/lib/lm_eval/tasks/aexams/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,247 copying build/lib/lm_eval/tasks/aexams/_aexams.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,249 copying build/lib/lm_eval/tasks/aexams/aexams_Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,251 copying build/lib/lm_eval/tasks/aexams/aexams_Science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,253 copying build/lib/lm_eval/tasks/aexams/aexams_IslamicStudies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,255 copying build/lib/lm_eval/tasks/aexams/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,257 copying build/lib/lm_eval/tasks/aexams/aexams_Social.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,258 copying build/lib/lm_eval/tasks/aexams/aexams_Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aexams 2026-05-11T15:52:58,261 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/translation 2026-05-11T15:52:58,262 copying build/lib/lm_eval/tasks/translation/wmt16_de-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,264 copying build/lib/lm_eval/tasks/translation/wmt16_en-ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,266 copying build/lib/lm_eval/tasks/translation/iwslt2017_ar-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,268 copying build/lib/lm_eval/tasks/translation/wmt14_fr-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,271 copying build/lib/lm_eval/tasks/translation/iwslt2017_en-ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,273 copying build/lib/lm_eval/tasks/translation/wmt_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,276 copying build/lib/lm_eval/tasks/translation/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,278 copying build/lib/lm_eval/tasks/translation/wmt16_ro-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,281 copying build/lib/lm_eval/tasks/translation/wmt14_en-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,283 copying build/lib/lm_eval/tasks/translation/wmt16_en-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,285 copying build/lib/lm_eval/tasks/translation/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/translation 2026-05-11T15:52:58,289 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mutual 2026-05-11T15:52:58,290 copying build/lib/lm_eval/tasks/mutual/mutual.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T15:52:58,293 copying build/lib/lm_eval/tasks/mutual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T15:52:58,296 copying build/lib/lm_eval/tasks/mutual/multual_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T15:52:58,298 copying build/lib/lm_eval/tasks/mutual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mutual 2026-05-11T15:52:58,301 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu 2026-05-11T15:52:58,304 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,306 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,308 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,310 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,313 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,315 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,317 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,319 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,322 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,324 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,326 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,328 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,330 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,333 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,335 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,337 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,340 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,342 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,344 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,346 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,348 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,351 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,353 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,355 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,358 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,360 copying build/lib/lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,362 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,365 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,367 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,369 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,370 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,373 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,375 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,378 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,380 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,381 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,383 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,385 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,387 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,389 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,390 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,392 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,394 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,396 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,398 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,399 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,401 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,403 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,405 copying build/lib/lm_eval/tasks/kmmlu/hard/_kmmlu_hard_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,407 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,409 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,411 copying build/lib/lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/hard 2026-05-11T15:52:58,414 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,415 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,418 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,420 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,422 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,424 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,426 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,429 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,432 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,434 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,436 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,439 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,441 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,444 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,446 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,449 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,451 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,453 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,455 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,458 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,460 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,463 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,465 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,467 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,470 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,472 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,475 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,477 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,479 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,482 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,484 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,486 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,489 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,492 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,494 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,497 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,499 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,501 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,504 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,506 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,509 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,511 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,513 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,516 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,518 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,521 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,523 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,525 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,528 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,530 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,533 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,535 copying build/lib/lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/cot_hard 2026-05-11T15:52:58,537 copying build/lib/lm_eval/tasks/kmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu 2026-05-11T15:52:58,541 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,542 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,544 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,545 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,547 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,549 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,551 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,553 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,555 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,557 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,559 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,561 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,563 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,565 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,566 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,568 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,570 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,572 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,574 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,576 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,578 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,579 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,581 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,583 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,585 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,587 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,589 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,590 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,592 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,594 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,596 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,598 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,600 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,602 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,604 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,606 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,608 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,610 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,612 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,614 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,615 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,617 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,619 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,621 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,623 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,624 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,626 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,628 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,630 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,632 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,634 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,635 copying build/lib/lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct_hard 2026-05-11T15:52:58,639 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,640 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,642 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,644 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,645 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,647 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,649 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,651 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,653 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,655 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,656 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,658 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,660 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,662 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,664 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,666 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,667 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,669 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,671 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,672 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,674 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,676 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,678 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,680 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,681 copying build/lib/lm_eval/tasks/kmmlu/default/_default_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,683 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,685 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,687 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,689 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,690 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,692 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,694 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,696 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,698 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,700 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,702 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,703 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,705 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,707 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,709 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,711 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,713 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,715 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,717 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,718 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,720 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,722 copying build/lib/lm_eval/tasks/kmmlu/default/_kmmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,724 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,725 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,727 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,729 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,731 copying build/lib/lm_eval/tasks/kmmlu/default/kmmlu_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/default 2026-05-11T15:52:58,734 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,735 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,738 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,739 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,741 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,743 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,745 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,747 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,749 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,751 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,753 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,755 copying build/lib/lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,756 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,758 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,760 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,762 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,763 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,765 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,767 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,769 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,771 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,772 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,774 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,776 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,778 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,780 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,782 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,783 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,785 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,787 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_humss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,789 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,791 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,793 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,795 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,797 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,799 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,801 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,802 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,804 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,806 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,807 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,809 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,811 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,813 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,815 copying build/lib/lm_eval/tasks/kmmlu/direct/_kmmlu_direct_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,816 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,818 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,820 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,822 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,824 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,825 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,827 copying build/lib/lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kmmlu/direct 2026-05-11T15:52:58,830 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swde 2026-05-11T15:52:58,831 copying build/lib/lm_eval/tasks/swde/swde.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swde 2026-05-11T15:52:58,833 copying build/lib/lm_eval/tasks/swde/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swde 2026-05-11T15:52:58,835 copying build/lib/lm_eval/tasks/swde/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swde 2026-05-11T15:52:58,838 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,839 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,841 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,843 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,845 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,847 copying build/lib/lm_eval/tasks/lambada_multilingual/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,849 copying build/lib/lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual 2026-05-11T15:52:58,853 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/belebele 2026-05-11T15:52:58,854 copying build/lib/lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,856 copying build/lib/lm_eval/tasks/belebele/belebele_kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,858 copying build/lib/lm_eval/tasks/belebele/belebele_mri_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,860 copying build/lib/lm_eval/tasks/belebele/belebele_als_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,862 copying build/lib/lm_eval/tasks/belebele/belebele_swe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,863 copying build/lib/lm_eval/tasks/belebele/belebele_sin_Sinh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,865 copying build/lib/lm_eval/tasks/belebele/belebele_ory_Orya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,867 copying build/lib/lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,869 copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,871 copying build/lib/lm_eval/tasks/belebele/belebele_tgl_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,873 copying build/lib/lm_eval/tasks/belebele/belebele_ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,875 copying build/lib/lm_eval/tasks/belebele/belebele_bod_Tibt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,876 copying build/lib/lm_eval/tasks/belebele/belebele_arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,878 copying build/lib/lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,880 copying build/lib/lm_eval/tasks/belebele/belebele_deu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,882 copying build/lib/lm_eval/tasks/belebele/belebele_asm_Beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,884 copying build/lib/lm_eval/tasks/belebele/belebele_fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,886 copying build/lib/lm_eval/tasks/belebele/belebele_tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,887 copying build/lib/lm_eval/tasks/belebele/belebele_apc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,889 copying build/lib/lm_eval/tasks/belebele/belebele_sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,891 copying build/lib/lm_eval/tasks/belebele/belebele_por_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,893 copying build/lib/lm_eval/tasks/belebele/belebele_arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,896 copying build/lib/lm_eval/tasks/belebele/belebele_tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,898 copying build/lib/lm_eval/tasks/belebele/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,900 copying build/lib/lm_eval/tasks/belebele/belebele_ckb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,902 copying build/lib/lm_eval/tasks/belebele/belebele_cat_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,903 copying build/lib/lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,905 copying build/lib/lm_eval/tasks/belebele/belebele_swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,907 copying build/lib/lm_eval/tasks/belebele/belebele_ilo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,909 copying build/lib/lm_eval/tasks/belebele/belebele_arb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,911 copying build/lib/lm_eval/tasks/belebele/belebele_kor_Hang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,912 copying build/lib/lm_eval/tasks/belebele/belebele_plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,914 copying build/lib/lm_eval/tasks/belebele/belebele_uzn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,916 copying build/lib/lm_eval/tasks/belebele/belebele_lvs_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,918 copying build/lib/lm_eval/tasks/belebele/belebele_shn_Mymr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,920 copying build/lib/lm_eval/tasks/belebele/belebele_ell_Grek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,921 copying build/lib/lm_eval/tasks/belebele/belebele_heb_Hebr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,923 copying build/lib/lm_eval/tasks/belebele/belebele_slv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,925 copying build/lib/lm_eval/tasks/belebele/belebele_snd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,927 copying build/lib/lm_eval/tasks/belebele/belebele_pol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,929 copying build/lib/lm_eval/tasks/belebele/belebele_hin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,931 copying build/lib/lm_eval/tasks/belebele/belebele_urd_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,932 copying build/lib/lm_eval/tasks/belebele/belebele_zsm_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,934 copying build/lib/lm_eval/tasks/belebele/belebele_hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,936 copying build/lib/lm_eval/tasks/belebele/belebele_urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,938 copying build/lib/lm_eval/tasks/belebele/belebele_est_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,940 copying build/lib/lm_eval/tasks/belebele/belebele_ben_Beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,942 copying build/lib/lm_eval/tasks/belebele/belebele_pes_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,944 copying build/lib/lm_eval/tasks/belebele/belebele_som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,946 copying build/lib/lm_eval/tasks/belebele/belebele_guj_Gujr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,948 copying build/lib/lm_eval/tasks/belebele/belebele_pan_Guru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,950 copying build/lib/lm_eval/tasks/belebele/belebele_fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,952 copying build/lib/lm_eval/tasks/belebele/belebele_fin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,954 copying build/lib/lm_eval/tasks/belebele/belebele_hun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,955 copying build/lib/lm_eval/tasks/belebele/belebele_azj_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,957 copying build/lib/lm_eval/tasks/belebele/belebele_ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,959 copying build/lib/lm_eval/tasks/belebele/belebele_ron_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,961 copying build/lib/lm_eval/tasks/belebele/belebele_tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,963 copying build/lib/lm_eval/tasks/belebele/belebele_isl_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,965 copying build/lib/lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,967 copying build/lib/lm_eval/tasks/belebele/belebele_eus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,968 copying build/lib/lm_eval/tasks/belebele/belebele_hat_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,970 copying build/lib/lm_eval/tasks/belebele/belebele_wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,972 copying build/lib/lm_eval/tasks/belebele/belebele_afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,974 copying build/lib/lm_eval/tasks/belebele/belebele_npi_Deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,976 copying build/lib/lm_eval/tasks/belebele/belebele_jav_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,978 copying build/lib/lm_eval/tasks/belebele/belebele_mya_Mymr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,980 copying build/lib/lm_eval/tasks/belebele/belebele_nld_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,981 copying build/lib/lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,983 copying build/lib/lm_eval/tasks/belebele/belebele_npi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,985 copying build/lib/lm_eval/tasks/belebele/belebele_nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,987 copying build/lib/lm_eval/tasks/belebele/belebele_kac_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,989 copying build/lib/lm_eval/tasks/belebele/belebele_ind_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,991 copying build/lib/lm_eval/tasks/belebele/belebele_luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,993 copying build/lib/lm_eval/tasks/belebele/belebele_amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,995 copying build/lib/lm_eval/tasks/belebele/belebele_zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,996 copying build/lib/lm_eval/tasks/belebele/belebele_lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:58,998 copying build/lib/lm_eval/tasks/belebele/belebele_tur_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,000 copying build/lib/lm_eval/tasks/belebele/belebele_pbt_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,002 copying build/lib/lm_eval/tasks/belebele/belebele_nob_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,004 copying build/lib/lm_eval/tasks/belebele/belebele_lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,006 copying build/lib/lm_eval/tasks/belebele/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,008 copying build/lib/lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,010 copying build/lib/lm_eval/tasks/belebele/belebele_ars_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,012 copying build/lib/lm_eval/tasks/belebele/belebele_lao_Laoo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,014 copying build/lib/lm_eval/tasks/belebele/belebele_nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,016 copying build/lib/lm_eval/tasks/belebele/belebele_kan_Knda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,017 copying build/lib/lm_eval/tasks/belebele/belebele_slk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,019 copying build/lib/lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,021 copying build/lib/lm_eval/tasks/belebele/belebele_hye_Armn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,023 copying build/lib/lm_eval/tasks/belebele/belebele_sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,025 copying build/lib/lm_eval/tasks/belebele/belebele_bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,027 copying build/lib/lm_eval/tasks/belebele/belebele_ita_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,029 copying build/lib/lm_eval/tasks/belebele/belebele_hin_Deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,030 copying build/lib/lm_eval/tasks/belebele/belebele_acm_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,032 copying build/lib/lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,034 copying build/lib/lm_eval/tasks/belebele/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,036 copying build/lib/lm_eval/tasks/belebele/belebele_hrv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,038 copying build/lib/lm_eval/tasks/belebele/belebele_sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,040 copying build/lib/lm_eval/tasks/belebele/belebele_grn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,042 copying build/lib/lm_eval/tasks/belebele/belebele_ces_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,044 copying build/lib/lm_eval/tasks/belebele/belebele_dan_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,046 copying build/lib/lm_eval/tasks/belebele/belebele_yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,047 copying build/lib/lm_eval/tasks/belebele/belebele_mal_Mlym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,049 copying build/lib/lm_eval/tasks/belebele/belebele_tha_Thai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,051 copying build/lib/lm_eval/tasks/belebele/belebele_xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,053 copying build/lib/lm_eval/tasks/belebele/belebele_vie_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,055 copying build/lib/lm_eval/tasks/belebele/belebele_mar_Deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,057 copying build/lib/lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,059 copying build/lib/lm_eval/tasks/belebele/belebele_ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,061 copying build/lib/lm_eval/tasks/belebele/belebele_tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,063 copying build/lib/lm_eval/tasks/belebele/belebele_spa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,064 copying build/lib/lm_eval/tasks/belebele/belebele_sin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,066 copying build/lib/lm_eval/tasks/belebele/belebele_tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,068 copying build/lib/lm_eval/tasks/belebele/belebele_kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,070 copying build/lib/lm_eval/tasks/belebele/_belebele.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,072 copying build/lib/lm_eval/tasks/belebele/belebele_kat_Geor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,074 copying build/lib/lm_eval/tasks/belebele/belebele_ben_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,077 copying build/lib/lm_eval/tasks/belebele/belebele_khm_Khmr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,079 copying build/lib/lm_eval/tasks/belebele/belebele_gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,081 copying build/lib/lm_eval/tasks/belebele/belebele_mlt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,084 copying build/lib/lm_eval/tasks/belebele/belebele_war_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,086 copying build/lib/lm_eval/tasks/belebele/belebele_zho_Hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,088 copying build/lib/lm_eval/tasks/belebele/belebele_eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,091 copying build/lib/lm_eval/tasks/belebele/belebele_ceb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,093 copying build/lib/lm_eval/tasks/belebele/belebele_lit_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/belebele 2026-05-11T15:52:59,096 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/winogrande 2026-05-11T15:52:59,098 copying build/lib/lm_eval/tasks/winogrande/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogrande 2026-05-11T15:52:59,101 copying build/lib/lm_eval/tasks/winogrande/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogrande 2026-05-11T15:52:59,103 copying build/lib/lm_eval/tasks/winogrande/preprocess_winogrande.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/winogrande 2026-05-11T15:52:59,106 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/chartqa 2026-05-11T15:52:59,108 copying build/lib/lm_eval/tasks/chartqa/chartqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T15:52:59,110 copying build/lib/lm_eval/tasks/chartqa/chartqa_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T15:52:59,113 copying build/lib/lm_eval/tasks/chartqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T15:52:59,115 copying build/lib/lm_eval/tasks/chartqa/chartqa_llama_90.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T15:52:59,117 copying build/lib/lm_eval/tasks/chartqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/chartqa 2026-05-11T15:52:59,122 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,123 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,125 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,127 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,130 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,132 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,134 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,136 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,139 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,141 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,143 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,145 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,148 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,150 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,152 copying build/lib/lm_eval/tasks/crows_pairs/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,155 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,158 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,160 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,162 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,164 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,167 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,169 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,171 copying build/lib/lm_eval/tasks/crows_pairs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,174 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,176 copying build/lib/lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/crows_pairs 2026-05-11T15:52:59,178 copying build/lib/lm_eval/tasks/_yaml_loader.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T15:52:59,181 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/click 2026-05-11T15:52:59,182 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,183 copying build/lib/lm_eval/tasks/click/click_lang/click_lang_function.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,185 copying build/lib/lm_eval/tasks/click/click_lang/click_lang_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,187 copying build/lib/lm_eval/tasks/click/click_lang/_click_lang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,189 copying build/lib/lm_eval/tasks/click/click_lang/click_lang_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,191 copying build/lib/lm_eval/tasks/click/click_lang/_default_click_lang_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,192 copying build/lib/lm_eval/tasks/click/click_lang/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_lang 2026-05-11T15:52:59,195 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,196 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,198 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_society.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,200 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_kpop.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,202 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,203 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_economy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,205 copying build/lib/lm_eval/tasks/click/click_cul/_click_cul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,207 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_tradition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,209 copying build/lib/lm_eval/tasks/click/click_cul/_default_click_cul_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,211 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,213 copying build/lib/lm_eval/tasks/click/click_cul/click_cul_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,214 copying build/lib/lm_eval/tasks/click/click_cul/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click/click_cul 2026-05-11T15:52:59,216 copying build/lib/lm_eval/tasks/click/click.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click 2026-05-11T15:52:59,218 copying build/lib/lm_eval/tasks/click/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/click 2026-05-11T15:52:59,221 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kobest 2026-05-11T15:52:59,222 copying build/lib/lm_eval/tasks/kobest/_kobest.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,224 copying build/lib/lm_eval/tasks/kobest/kobest_wic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,226 copying build/lib/lm_eval/tasks/kobest/kobest_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,227 copying build/lib/lm_eval/tasks/kobest/kobest_boolq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,229 copying build/lib/lm_eval/tasks/kobest/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,231 copying build/lib/lm_eval/tasks/kobest/kobest_sentineg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,233 copying build/lib/lm_eval/tasks/kobest/kobest_copa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,235 copying build/lib/lm_eval/tasks/kobest/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kobest 2026-05-11T15:52:59,237 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squad_completion 2026-05-11T15:52:59,238 copying build/lib/lm_eval/tasks/squad_completion/squad_completion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squad_completion 2026-05-11T15:52:59,240 copying build/lib/lm_eval/tasks/squad_completion/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squad_completion 2026-05-11T15:52:59,242 copying build/lib/lm_eval/tasks/squad_completion/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squad_completion 2026-05-11T15:52:59,247 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,248 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_prepP.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,250 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,252 copying build/lib/lm_eval/tasks/zhoblimp/question_V_not_VP_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,253 copying build/lib/lm_eval/tasks/zhoblimp/BA_suo_adverbial_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,255 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,257 copying build/lib/lm_eval/tasks/zhoblimp/zhoblimp_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,260 copying build/lib/lm_eval/tasks/zhoblimp/BA_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,262 copying build/lib/lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,263 copying build/lib/lm_eval/tasks/zhoblimp/question_particle_nandao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,265 copying build/lib/lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,267 copying build/lib/lm_eval/tasks/zhoblimp/PN_numP_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,269 copying build/lib/lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,271 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,273 copying build/lib/lm_eval/tasks/zhoblimp/noun_adjective_shi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,275 copying build/lib/lm_eval/tasks/zhoblimp/anaphor_number_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,277 copying build/lib/lm_eval/tasks/zhoblimp/right_yijing_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,279 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_wh_question_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,281 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,283 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_suoyou.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,284 copying build/lib/lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,287 copying build/lib/lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,289 copying build/lib/lm_eval/tasks/zhoblimp/intransitive_double_obj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,290 copying build/lib/lm_eval/tasks/zhoblimp/you_yige.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,292 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A_daodi_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,294 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,296 copying build/lib/lm_eval/tasks/zhoblimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,298 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_d.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,300 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_locP.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,301 copying build/lib/lm_eval/tasks/zhoblimp/classifier_noun_agreement_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,303 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_1_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,305 copying build/lib/lm_eval/tasks/zhoblimp/control_modal_vs_raising_modal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,307 copying build/lib/lm_eval/tasks/zhoblimp/ellipsis_n_bar_class.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,309 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,310 copying build/lib/lm_eval/tasks/zhoblimp/PN_numP_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,312 copying build/lib/lm_eval/tasks/zhoblimp/BEI_construction_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,314 copying build/lib/lm_eval/tasks/zhoblimp/BA_no_progressive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,316 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,318 copying build/lib/lm_eval/tasks/zhoblimp/anaphor_gender_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,320 copying build/lib/lm_eval/tasks/zhoblimp/relative_operator_intepretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,322 copying build/lib/lm_eval/tasks/zhoblimp/ya_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,323 copying build/lib/lm_eval/tasks/zhoblimp/ellipsis_adj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,325 copying build/lib/lm_eval/tasks/zhoblimp/BA_meiba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,327 copying build/lib/lm_eval/tasks/zhoblimp/question_particle_daodi_choice_tran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,329 copying build/lib/lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,331 copying build/lib/lm_eval/tasks/zhoblimp/classifier_noun_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,333 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_OSV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,334 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_1_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,336 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,338 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_conditional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,340 copying build/lib/lm_eval/tasks/zhoblimp/left_dou.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,342 copying build/lib/lm_eval/tasks/zhoblimp/BA_no_stative_verb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,344 copying build/lib/lm_eval/tasks/zhoblimp/adjective_transitive_dui.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,346 copying build/lib/lm_eval/tasks/zhoblimp/BEI_preposition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,348 copying build/lib/lm_eval/tasks/zhoblimp/modal_raising_topicalization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,350 copying build/lib/lm_eval/tasks/zhoblimp/ellipsis_double_object.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,351 copying build/lib/lm_eval/tasks/zhoblimp/BA_inversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,353 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_tran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,355 copying build/lib/lm_eval/tasks/zhoblimp/preposition_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,357 copying build/lib/lm_eval/tasks/zhoblimp/BA_verb_le_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,359 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A_indirect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,361 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_A_not_A_question.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,363 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_domain_number.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,365 copying build/lib/lm_eval/tasks/zhoblimp/causative_shi_ba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,367 copying build/lib/lm_eval/tasks/zhoblimp/verb_negation_particle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,369 copying build/lib/lm_eval/tasks/zhoblimp/BA_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,371 copying build/lib/lm_eval/tasks/zhoblimp/passive_no_adj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,373 copying build/lib/lm_eval/tasks/zhoblimp/question_particle_daodi_choice_intran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,374 copying build/lib/lm_eval/tasks/zhoblimp/BA_verb_le_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,376 copying build/lib/lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,378 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,380 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_scope_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,382 copying build/lib/lm_eval/tasks/zhoblimp/plural_cardinal_men_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,384 copying build/lib/lm_eval/tasks/zhoblimp/agent_animacy_subj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,386 copying build/lib/lm_eval/tasks/zhoblimp/passive_body_part.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,388 copying build/lib/lm_eval/tasks/zhoblimp/modal_raising_hui.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,390 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,391 copying build/lib/lm_eval/tasks/zhoblimp/npi_renhe_wh_question_obj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,393 copying build/lib/lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,395 copying build/lib/lm_eval/tasks/zhoblimp/agent_animacy_passive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,397 copying build/lib/lm_eval/tasks/zhoblimp/question_A_not_A_daodi_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,399 copying build/lib/lm_eval/tasks/zhoblimp/plural_cardinal_men_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,401 copying build/lib/lm_eval/tasks/zhoblimp/question_V_not_VP_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,402 copying build/lib/lm_eval/tasks/zhoblimp/intransitive_no_obj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,404 copying build/lib/lm_eval/tasks/zhoblimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,406 copying build/lib/lm_eval/tasks/zhoblimp/agent_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,408 copying build/lib/lm_eval/tasks/zhoblimp/nominal_definite_men.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,410 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_intran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,412 copying build/lib/lm_eval/tasks/zhoblimp/BA_BEI_subj_drop.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,414 copying build/lib/lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,416 copying build/lib/lm_eval/tasks/zhoblimp/nominal_modal_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,417 copying build/lib/lm_eval/tasks/zhoblimp/preposition_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,419 copying build/lib/lm_eval/tasks/zhoblimp/BA_suo_adverbial_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,421 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_ruguo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,423 copying build/lib/lm_eval/tasks/zhoblimp/left_adverbial_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,425 copying build/lib/lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,427 copying build/lib/lm_eval/tasks/zhoblimp/relative_operator_who.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,429 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,431 copying build/lib/lm_eval/tasks/zhoblimp/passive_suo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,433 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,434 copying build/lib/lm_eval/tasks/zhoblimp/right_yijing_b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,436 copying build/lib/lm_eval/tasks/zhoblimp/BEI_construction_a.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,438 copying build/lib/lm_eval/tasks/zhoblimp/question_daodi_nandao_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,440 copying build/lib/lm_eval/tasks/zhoblimp/agent_causative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,442 copying build/lib/lm_eval/tasks/zhoblimp/topicalization_SOV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,443 copying build/lib/lm_eval/tasks/zhoblimp/classifier_noun_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,445 copying build/lib/lm_eval/tasks/zhoblimp/BEI_deletion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,447 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_long_left.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,449 copying build/lib/lm_eval/tasks/zhoblimp/noun_phrase_conjunction_jian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,451 copying build/lib/lm_eval/tasks/zhoblimp/fci_renhe_dou.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,452 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_c_command_number.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,454 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_domain.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,456 copying build/lib/lm_eval/tasks/zhoblimp/agent_animacy_adv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,458 copying build/lib/lm_eval/tasks/zhoblimp/BA_duplicate_argument.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,460 copying build/lib/lm_eval/tasks/zhoblimp/existential_there_subject_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,462 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_raising_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,464 copying build/lib/lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,466 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_scope_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,468 copying build/lib/lm_eval/tasks/zhoblimp/passive_intransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,470 copying build/lib/lm_eval/tasks/zhoblimp/passive_agent_deletion_short.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,471 copying build/lib/lm_eval/tasks/zhoblimp/question_nandao_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,473 copying build/lib/lm_eval/tasks/zhoblimp/principle_A_c_command.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/zhoblimp 2026-05-11T15:52:59,476 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wsc273 2026-05-11T15:52:59,477 copying build/lib/lm_eval/tasks/wsc273/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wsc273 2026-05-11T15:52:59,479 copying build/lib/lm_eval/tasks/wsc273/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wsc273 2026-05-11T15:52:59,481 copying build/lib/lm_eval/tasks/wsc273/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wsc273 2026-05-11T15:52:59,484 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unitxt 2026-05-11T15:52:59,485 copying build/lib/lm_eval/tasks/unitxt/law_stack_exchange.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,487 copying build/lib/lm_eval/tasks/unitxt/medical_abstracts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,489 copying build/lib/lm_eval/tasks/unitxt/claim_stance_topic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,491 copying build/lib/lm_eval/tasks/unitxt/unitxt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,493 copying build/lib/lm_eval/tasks/unitxt/yahoo_answers_topics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,494 copying build/lib/lm_eval/tasks/unitxt/ag_news.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,496 copying build/lib/lm_eval/tasks/unitxt/ledgar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,498 copying build/lib/lm_eval/tasks/unitxt/banking77.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,500 copying build/lib/lm_eval/tasks/unitxt/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,502 copying build/lib/lm_eval/tasks/unitxt/financial_tweets.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,504 copying build/lib/lm_eval/tasks/unitxt/20_newsgroups.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,506 copying build/lib/lm_eval/tasks/unitxt/cnn_dailymail.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,507 copying build/lib/lm_eval/tasks/unitxt/stsb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,509 copying build/lib/lm_eval/tasks/unitxt/xsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,511 copying build/lib/lm_eval/tasks/unitxt/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,513 copying build/lib/lm_eval/tasks/unitxt/unfair_tos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,515 copying build/lib/lm_eval/tasks/unitxt/dbpedia_14.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,517 copying build/lib/lm_eval/tasks/unitxt/atis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,519 copying build/lib/lm_eval/tasks/unitxt/argument_topic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,521 copying build/lib/lm_eval/tasks/unitxt/doc_vqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,522 copying build/lib/lm_eval/tasks/unitxt/coedit_gec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,524 copying build/lib/lm_eval/tasks/unitxt/ethos_binary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,526 copying build/lib/lm_eval/tasks/unitxt/unitxt_multimodal -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unitxt 2026-05-11T15:52:59,528 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light 2026-05-11T15:52:59,529 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light 2026-05-11T15:52:59,532 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:59,533 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_mt_hellaswag_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:59,535 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_leaderboard_arabic_mt_hellaswag_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:59,537 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light 2026-05-11T15:52:59,541 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,542 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_world_religions_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,544 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_sexuality_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,546 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_jurisprudence_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,548 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_business_ethics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,550 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_security_studies_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,552 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_scenarios_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,554 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_aging_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,556 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_marketing_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,558 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_global_facts_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,560 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_biology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,562 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_public_relations_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,563 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_elementary_mathematics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,565 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_econometrics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,567 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_computer_science_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,569 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_computer_science_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,571 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_international_law_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,573 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_accounting_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,575 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_us_foreign_policy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,577 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_psychology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,579 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,580 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_geography_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,582 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_virology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,584 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_computer_security_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,586 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_us_history_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,588 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_law_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,590 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_logical_fallacies_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,592 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_formal_logic_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,594 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_anatomy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,596 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_disputes_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,598 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_biology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,599 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_mathematics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,601 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_chemistry_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,603 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_conceptual_physics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,605 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_clinical_knowledge_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,607 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_prehistory_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,609 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_physics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,611 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_sociology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,613 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_nutrition_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,615 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,617 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_philosophy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,620 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_mathematics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,622 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_world_history_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,624 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_statistics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,626 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_abstract_algebra_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,628 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_management_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,629 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_european_history_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,631 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_machine_learning_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,633 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_astronomy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,635 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_psychology_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,637 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,639 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_medicine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,641 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_physics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,643 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_medicine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,645 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_chemistry_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,647 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_medical_genetics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,648 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_miscellaneous_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,650 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_electrical_engineering_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,652 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,654 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_microeconomics_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light 2026-05-11T15:52:59,657 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:59,658 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_leaderboard_arabic_exams_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:59,660 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_exams_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:59,661 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light 2026-05-11T15:52:59,664 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:59,665 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_leaderboard_arabic_mt_race_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:59,667 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_mt_race_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:59,669 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light 2026-05-11T15:52:59,673 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,674 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Bahrain_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,676 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Geography_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,677 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Syria_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,679 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Libya_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,682 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Qatar_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,684 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_United_Arab_Emirates_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,685 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Funeral_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,687 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Food_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,689 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Saudi_Arabia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,691 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Egypt_modern_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,693 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromIslam_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,695 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromByzantium_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,697 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Yemen_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,699 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islamic_law_system_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,701 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Language_Origin_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,703 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromPersia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,705 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,707 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Music_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,709 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mauritania_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,710 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Palestine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,712 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Literature_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,714 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_Education_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,716 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Tunisia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,718 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Comoros_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,719 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_computer_and_phone_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,722 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_daily_life_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,723 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Ancient_Egypt_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,725 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arab_Empire_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,727 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Oman_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,729 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_History_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,731 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Lebanon_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,733 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Medicine_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,735 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Art_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,737 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Morocco_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,739 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Algeria_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,741 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromAncientEgypt_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,742 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Clothing_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,744 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Iraq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,746 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ornament_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,748 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Astronomy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,750 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mesopotamia_civilization_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,752 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromRome_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,754 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Math_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,755 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Wedding_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,758 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_communication_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,759 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_branches_and_schools_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,761 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Jordan_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,763 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Philosophy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,765 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ceremony_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,767 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Somalia_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,769 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromGreece_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,771 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Architecture_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,774 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Calligraphy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,775 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,777 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Kuwait_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,779 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromChina_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,781 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Culture_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,783 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,785 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_entertainment_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,787 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Sudan_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light 2026-05-11T15:52:59,789 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:59,791 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_leaderboard_arabic_mt_toxigen_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:59,793 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_mt_toxigen_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:59,795 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light 2026-05-11T15:52:59,797 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:59,798 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arabic_mt_copa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:59,801 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arbic_leaderboard_arabic_mt_copa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:59,802 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light 2026-05-11T15:52:59,805 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:59,806 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_leaderboard_arabic_mt_boolq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:59,808 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_mt_boolq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:59,809 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light 2026-05-11T15:52:59,812 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:59,813 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_leaderboard_arabic_mt_openbook_qa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:59,815 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_mt_openbook_qa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:59,817 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light 2026-05-11T15:52:59,819 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:59,820 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_mt_mmlu_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:59,822 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:59,824 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_leaderboard_arabic_mt_mmlu_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light 2026-05-11T15:52:59,826 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light 2026-05-11T15:52:59,828 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:59,830 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_leaderboard_arabic_mt_arc_easy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:59,832 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_mt_arc_easy_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:59,834 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light 2026-05-11T15:52:59,836 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:59,837 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_leaderboard_arabic_mt_piqa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:59,839 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_mt_piqa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:59,841 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light 2026-05-11T15:52:59,843 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:59,844 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_leaderboard_arabic_mt_sciq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:59,846 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:59,848 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_mt_sciq_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light 2026-05-11T15:52:59,850 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:59,851 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_mt_arc_challenge_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:59,853 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_leaderboard_arabic_mt_arc_challenge_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:59,855 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light 2026-05-11T15:52:59,857 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,858 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,860 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,862 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_msa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,864 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,866 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,868 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_dialects_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,870 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,872 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_mcq_exams_test_ar_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,873 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,875 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_sentiment_task_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,877 copying build/lib/lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light 2026-05-11T15:52:59,880 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/minerva_math 2026-05-11T15:52:59,882 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,884 copying build/lib/lm_eval/tasks/minerva_math/minerva_math500.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,886 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,889 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,891 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,893 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,895 copying build/lib/lm_eval/tasks/minerva_math/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,898 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,900 copying build/lib/lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,903 copying build/lib/lm_eval/tasks/minerva_math/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/minerva_math 2026-05-11T15:52:59,906 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hellaswag 2026-05-11T15:52:59,907 copying build/lib/lm_eval/tasks/hellaswag/hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hellaswag 2026-05-11T15:52:59,910 copying build/lib/lm_eval/tasks/hellaswag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hellaswag 2026-05-11T15:52:59,912 copying build/lib/lm_eval/tasks/hellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hellaswag 2026-05-11T15:52:59,915 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,916 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,918 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,921 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,923 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,925 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,927 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,930 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,932 copying build/lib/lm_eval/tasks/lambada_multilingual_stablelm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_multilingual_stablelm 2026-05-11T15:52:59,935 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score 2026-05-11T15:52:59,937 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,939 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_aqua_rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,941 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lsat_rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,943 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_aqua_rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,946 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,948 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,951 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,953 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_logiqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,956 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_logiqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,958 copying build/lib/lm_eval/tasks/score/agi_eval/score_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,960 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,963 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,965 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_aqua_rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,967 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,969 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,972 copying build/lib/lm_eval/tasks/score/agi_eval/score_option_order_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,974 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,976 copying build/lib/lm_eval/tasks/score/agi_eval/score_non_greedy_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,979 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_logiqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,980 copying build/lib/lm_eval/tasks/score/agi_eval/score_prompt_robustness_agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,982 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,984 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,986 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,988 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_templates.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,991 copying build/lib/lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,993 copying build/lib/lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lsat_rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,995 copying build/lib/lm_eval/tasks/score/agi_eval/utils_agieval.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,997 copying build/lib/lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/agi_eval 2026-05-11T15:52:59,999 copying build/lib/lm_eval/tasks/score/non_greedy_summarizer.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T15:53:00,002 copying build/lib/lm_eval/tasks/score/non_greedy.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T15:53:00,004 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score/math 2026-05-11T15:53:00,005 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,008 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,010 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,012 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,014 copying build/lib/lm_eval/tasks/score/math/to_be_fixed_questions.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,016 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,018 copying build/lib/lm_eval/tasks/score/math/score_non_greedy_robustness_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,020 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,022 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,024 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,026 copying build/lib/lm_eval/tasks/score/math/math_grader.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,029 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,031 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,033 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,035 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,036 copying build/lib/lm_eval/tasks/score/math/non_greedy_robustness_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,038 copying build/lib/lm_eval/tasks/score/math/score_prompt_robustness_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,040 copying build/lib/lm_eval/tasks/score/math/score_robustness_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,042 copying build/lib/lm_eval/tasks/score/math/prompt_templates.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,044 copying build/lib/lm_eval/tasks/score/math/prompt_robustness_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,046 copying build/lib/lm_eval/tasks/score/math/utils_math.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/math 2026-05-11T15:53:00,049 copying build/lib/lm_eval/tasks/score/NON_GREEDY.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T15:53:00,051 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/score/mmlu_pro 2026-05-11T15:53:00,052 copying build/lib/lm_eval/tasks/score/mmlu_pro/score_non_greedy_robustness_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T15:53:00,055 copying build/lib/lm_eval/tasks/score/mmlu_pro/score_option_order_robustness_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T15:53:00,057 copying build/lib/lm_eval/tasks/score/mmlu_pro/utils_mmlu_pro.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T15:53:00,059 copying build/lib/lm_eval/tasks/score/mmlu_pro/prompt_templates.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T15:53:00,061 copying build/lib/lm_eval/tasks/score/mmlu_pro/score_prompt_robustness_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score/mmlu_pro 2026-05-11T15:53:00,063 copying build/lib/lm_eval/tasks/score/score_robustness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T15:53:00,065 copying build/lib/lm_eval/tasks/score/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T15:53:00,068 copying build/lib/lm_eval/tasks/score/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/score 2026-05-11T15:53:00,070 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/realtoxicityprompts 2026-05-11T15:53:00,071 copying build/lib/lm_eval/tasks/realtoxicityprompts/metric.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/realtoxicityprompts 2026-05-11T15:53:00,074 copying build/lib/lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/realtoxicityprompts 2026-05-11T15:53:00,076 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/meqsum 2026-05-11T15:53:00,077 copying build/lib/lm_eval/tasks/meqsum/meqsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meqsum 2026-05-11T15:53:00,079 copying build/lib/lm_eval/tasks/meqsum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meqsum 2026-05-11T15:53:00,081 copying build/lib/lm_eval/tasks/meqsum/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meqsum 2026-05-11T15:53:00,084 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bear 2026-05-11T15:53:00,085 copying build/lib/lm_eval/tasks/bear/bear.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bear 2026-05-11T15:53:00,087 copying build/lib/lm_eval/tasks/bear/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bear 2026-05-11T15:53:00,089 copying build/lib/lm_eval/tasks/bear/bear_big.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bear 2026-05-11T15:53:00,091 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k 2026-05-11T15:53:00,092 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T15:53:00,094 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T15:53:00,096 copying build/lib/lm_eval/tasks/gsm8k/gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T15:53:00,098 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T15:53:00,100 copying build/lib/lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T15:53:00,102 copying build/lib/lm_eval/tasks/gsm8k/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k 2026-05-11T15:53:00,105 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/libra 2026-05-11T15:53:00,106 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,108 copying build/lib/lm_eval/tasks/libra/librusec_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,110 copying build/lib/lm_eval/tasks/libra/_question_answering_and_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,112 copying build/lib/lm_eval/tasks/libra/passkey_with_librusec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,114 copying build/lib/lm_eval/tasks/libra/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,116 copying build/lib/lm_eval/tasks/libra/ru_sci_passage_count.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,118 copying build/lib/lm_eval/tasks/libra/ru_2wikimultihopqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,120 copying build/lib/lm_eval/tasks/libra/ru_sci_abstract_retrieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,122 copying build/lib/lm_eval/tasks/libra/matreshka_yes_no.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,123 copying build/lib/lm_eval/tasks/libra/matreshka_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,126 copying build/lib/lm_eval/tasks/libra/_complex_reasoning_and_mathematical_problems.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,127 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,129 copying build/lib/lm_eval/tasks/libra/long_context_multiq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,131 copying build/lib/lm_eval/tasks/libra/ru_gsm100.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,133 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,134 copying build/lib/lm_eval/tasks/libra/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,136 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,139 copying build/lib/lm_eval/tasks/libra/ru_babilong_qa1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,141 copying build/lib/lm_eval/tasks/libra/librusec_mhqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,143 copying build/lib/lm_eval/tasks/libra/ru_qasper.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,145 copying build/lib/lm_eval/tasks/libra/passkey.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,146 copying build/lib/lm_eval/tasks/libra/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,149 copying build/lib/lm_eval/tasks/libra/_simple_information_retrieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,151 copying build/lib/lm_eval/tasks/libra/_multi_hop_question_answering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,152 copying build/lib/lm_eval/tasks/libra/ru_quality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/libra 2026-05-11T15:53:00,155 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/common_voice 2026-05-11T15:53:00,156 copying build/lib/lm_eval/tasks/common_voice/common_voice_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/common_voice 2026-05-11T15:53:00,158 copying build/lib/lm_eval/tasks/common_voice/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/common_voice 2026-05-11T15:53:00,161 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glianorex 2026-05-11T15:53:00,162 copying build/lib/lm_eval/tasks/glianorex/glianorex_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T15:53:00,164 copying build/lib/lm_eval/tasks/glianorex/glianorex_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T15:53:00,165 copying build/lib/lm_eval/tasks/glianorex/glianorex.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T15:53:00,167 copying build/lib/lm_eval/tasks/glianorex/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T15:53:00,169 copying build/lib/lm_eval/tasks/glianorex/preprocess_glianorex.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glianorex 2026-05-11T15:53:00,172 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,173 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,175 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,177 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,179 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,181 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math500.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,182 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,184 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,186 copying build/lib/lm_eval/tasks/hendrycks_math/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,188 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,190 copying build/lib/lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,191 copying build/lib/lm_eval/tasks/hendrycks_math/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_math 2026-05-11T15:53:00,195 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,196 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,198 copying build/lib/lm_eval/tasks/mmlu-pro-plus/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,199 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,201 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,203 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,205 copying build/lib/lm_eval/tasks/mmlu-pro-plus/_mmlu_pro_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,207 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,208 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,210 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,212 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,214 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,216 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,218 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,220 copying build/lib/lm_eval/tasks/mmlu-pro-plus/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,222 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,224 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,225 copying build/lib/lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,227 copying build/lib/lm_eval/tasks/mmlu-pro-plus/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-pro-plus 2026-05-11T15:53:00,230 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench 2026-05-11T15:53:00,231 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,233 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,235 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,237 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,239 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,241 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,242 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/_boolq_cot_2shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,244 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,246 copying build/lib/lm_eval/tasks/acpbench/boolq_cot_2shot/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/boolq_cot_2shot 2026-05-11T15:53:00,249 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,250 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/next_act.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,252 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/acp_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,255 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,257 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,259 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/_gen_yaml_2shot -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,260 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,262 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,264 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,266 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/acp_grammar.lark -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,268 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,270 copying build/lib/lm_eval/tasks/acpbench/gen_2shot/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot 2026-05-11T15:53:00,273 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,274 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/next_act.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,277 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,280 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,283 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,285 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/_gen_yaml_2shot -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,287 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,289 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,292 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,294 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_grammar.lark -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,296 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,299 copying build/lib/lm_eval/tasks/acpbench/gen_2shot_with_pddl/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/gen_2shot_with_pddl 2026-05-11T15:53:00,301 copying build/lib/lm_eval/tasks/acpbench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench 2026-05-11T15:53:00,304 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,305 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/val.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,307 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/act_reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,309 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/land.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,311 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/reach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,313 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/prog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,315 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/_mcq_cot_2shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,317 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/just.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,319 copying build/lib/lm_eval/tasks/acpbench/mcq_cot_2shot/app.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/acpbench/mcq_cot_2shot 2026-05-11T15:53:00,322 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,323 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_morocco.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,325 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_nile_valley.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,327 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,329 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,331 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_gulf.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,333 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,335 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_libya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,337 copying build/lib/lm_eval/tasks/arab_culture_completion/prompts.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,339 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_levant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,341 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_uae.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,343 copying build/lib/lm_eval/tasks/arab_culture_completion/_default_arab_culture_completion_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,345 copying build/lib/lm_eval/tasks/arab_culture_completion/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,347 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,348 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_sudan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,350 copying build/lib/lm_eval/tasks/arab_culture_completion/utils_completion.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,352 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_algeria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,354 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,356 copying build/lib/lm_eval/tasks/arab_culture_completion/_arab_culture_completion_north_africa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,358 copying build/lib/lm_eval/tasks/arab_culture_completion/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,360 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_tunisia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,362 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_yemen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,363 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,365 copying build/lib/lm_eval/tasks/arab_culture_completion/arab_culture_completion_ksa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arab_culture_completion 2026-05-11T15:53:00,368 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/storycloze 2026-05-11T15:53:00,369 copying build/lib/lm_eval/tasks/storycloze/storycloze_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/storycloze 2026-05-11T15:53:00,371 copying build/lib/lm_eval/tasks/storycloze/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/storycloze 2026-05-11T15:53:00,373 copying build/lib/lm_eval/tasks/storycloze/storycloze_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/storycloze 2026-05-11T15:53:00,376 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basque_bench 2026-05-11T15:53:00,377 copying build/lib/lm_eval/tasks/basque_bench/arc_eu_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,379 copying build/lib/lm_eval/tasks/basque_bench/arc_eu_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,381 copying build/lib/lm_eval/tasks/basque_bench/basque_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,382 copying build/lib/lm_eval/tasks/basque_bench/wnli_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,384 copying build/lib/lm_eval/tasks/basque_bench/mgsm_native_cot_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,386 copying build/lib/lm_eval/tasks/basque_bench/mgsm_direct_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,388 copying build/lib/lm_eval/tasks/basque_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,390 copying build/lib/lm_eval/tasks/basque_bench/xcopa_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,393 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,394 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_fr-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,396 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,397 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,399 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_gl-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,401 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/create_yamls_flores_eu.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,403 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_es-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,405 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_ca-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,407 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_en-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,409 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,411 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,413 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,415 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_pt-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,417 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,419 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,421 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,423 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,425 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_de-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,427 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_it-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,429 copying build/lib/lm_eval/tasks/basque_bench/flores_eu/flores_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench/flores_eu 2026-05-11T15:53:00,431 copying build/lib/lm_eval/tasks/basque_bench/paws_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,433 copying build/lib/lm_eval/tasks/basque_bench/piqa_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,434 copying build/lib/lm_eval/tasks/basque_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basque_bench 2026-05-11T15:53:00,438 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darijammlu 2026-05-11T15:53:00,439 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,440 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,442 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_management_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,444 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,446 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,448 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,450 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,451 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,453 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,455 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,457 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,459 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,461 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,462 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,464 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,466 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,468 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,470 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,472 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,474 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,476 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_political_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,478 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,480 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,481 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,483 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,485 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,487 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_arabic_language_(general).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,489 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,491 copying build/lib/lm_eval/tasks/darijammlu/_darijammlu_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,493 copying build/lib/lm_eval/tasks/darijammlu/_darijammlu_ar_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,494 copying build/lib/lm_eval/tasks/darijammlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,497 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,499 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,501 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,502 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_philosophy_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,504 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,506 copying build/lib/lm_eval/tasks/darijammlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,508 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_arabic_language_(grammar).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,510 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,512 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,514 copying build/lib/lm_eval/tasks/darijammlu/_darijammlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,515 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,517 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_natural_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,519 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,521 copying build/lib/lm_eval/tasks/darijammlu/_default_darijammlu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,523 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,525 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,527 copying build/lib/lm_eval/tasks/darijammlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,530 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,533 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,535 copying build/lib/lm_eval/tasks/darijammlu/darijammlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijammlu 2026-05-11T15:53:00,538 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/jsonschema_bench 2026-05-11T15:53:00,539 copying build/lib/lm_eval/tasks/jsonschema_bench/jsonschema_bench_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T15:53:00,541 copying build/lib/lm_eval/tasks/jsonschema_bench/jsonschema_bench_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T15:53:00,543 copying build/lib/lm_eval/tasks/jsonschema_bench/jsonschema_bench_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T15:53:00,546 copying build/lib/lm_eval/tasks/jsonschema_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T15:53:00,548 copying build/lib/lm_eval/tasks/jsonschema_bench/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/jsonschema_bench 2026-05-11T15:53:00,550 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/discrim_eval 2026-05-11T15:53:00,551 copying build/lib/lm_eval/tasks/discrim_eval/discrim_eval_implicit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T15:53:00,553 copying build/lib/lm_eval/tasks/discrim_eval/discrim_eval_explicit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T15:53:00,555 copying build/lib/lm_eval/tasks/discrim_eval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T15:53:00,557 copying build/lib/lm_eval/tasks/discrim_eval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/discrim_eval 2026-05-11T15:53:00,560 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue 2026-05-11T15:53:00,561 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/cb 2026-05-11T15:53:00,562 copying build/lib/lm_eval/tasks/super_glue/cb/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T15:53:00,564 copying build/lib/lm_eval/tasks/super_glue/cb/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T15:53:00,566 copying build/lib/lm_eval/tasks/super_glue/cb/aggregate.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T15:53:00,568 copying build/lib/lm_eval/tasks/super_glue/cb/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/cb 2026-05-11T15:53:00,571 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/copa 2026-05-11T15:53:00,572 copying build/lib/lm_eval/tasks/super_glue/copa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/copa 2026-05-11T15:53:00,574 copying build/lib/lm_eval/tasks/super_glue/copa/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/copa 2026-05-11T15:53:00,576 copying build/lib/lm_eval/tasks/super_glue/copa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/copa 2026-05-11T15:53:00,578 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wic 2026-05-11T15:53:00,580 copying build/lib/lm_eval/tasks/super_glue/wic/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wic 2026-05-11T15:53:00,582 copying build/lib/lm_eval/tasks/super_glue/wic/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wic 2026-05-11T15:53:00,584 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/record 2026-05-11T15:53:00,585 copying build/lib/lm_eval/tasks/super_glue/record/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T15:53:00,587 copying build/lib/lm_eval/tasks/super_glue/record/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T15:53:00,589 copying build/lib/lm_eval/tasks/super_glue/record/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T15:53:00,591 copying build/lib/lm_eval/tasks/super_glue/record/util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/record 2026-05-11T15:53:00,594 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/multirc 2026-05-11T15:53:00,595 copying build/lib/lm_eval/tasks/super_glue/multirc/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/multirc 2026-05-11T15:53:00,597 copying build/lib/lm_eval/tasks/super_glue/multirc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/multirc 2026-05-11T15:53:00,599 copying build/lib/lm_eval/tasks/super_glue/multirc/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/multirc 2026-05-11T15:53:00,601 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/rte 2026-05-11T15:53:00,602 copying build/lib/lm_eval/tasks/super_glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/rte 2026-05-11T15:53:00,604 copying build/lib/lm_eval/tasks/super_glue/rte/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/rte 2026-05-11T15:53:00,606 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/wsc 2026-05-11T15:53:00,607 copying build/lib/lm_eval/tasks/super_glue/wsc/preprocess_wsc.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T15:53:00,609 copying build/lib/lm_eval/tasks/super_glue/wsc/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T15:53:00,611 copying build/lib/lm_eval/tasks/super_glue/wsc/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T15:53:00,613 copying build/lib/lm_eval/tasks/super_glue/wsc/t5_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/wsc 2026-05-11T15:53:00,615 copying build/lib/lm_eval/tasks/super_glue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue 2026-05-11T15:53:00,617 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/super_glue/boolq 2026-05-11T15:53:00,618 copying build/lib/lm_eval/tasks/super_glue/boolq/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/boolq 2026-05-11T15:53:00,620 copying build/lib/lm_eval/tasks/super_glue/boolq/t5-prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/boolq 2026-05-11T15:53:00,622 copying build/lib/lm_eval/tasks/super_glue/boolq/seq2seq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/super_glue/boolq 2026-05-11T15:53:00,624 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hrm8k 2026-05-11T15:53:00,626 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,627 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_omni_math_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,629 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_mmmlu_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,631 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_ksm_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,636 copying build/lib/lm_eval/tasks/hrm8k/en/_hrm8k_en_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,638 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,640 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_gsm8k_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,642 copying build/lib/lm_eval/tasks/hrm8k/en/hrm8k_math_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,644 copying build/lib/lm_eval/tasks/hrm8k/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/en 2026-05-11T15:53:00,646 copying build/lib/lm_eval/tasks/hrm8k/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k 2026-05-11T15:53:00,649 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,650 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_ksm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,652 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,653 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_omni_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,655 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,657 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,659 copying build/lib/lm_eval/tasks/hrm8k/default/_hrm8k_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,661 copying build/lib/lm_eval/tasks/hrm8k/default/hrm8k_mmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,663 copying build/lib/lm_eval/tasks/hrm8k/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hrm8k/default 2026-05-11T15:53:00,666 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/toxigen 2026-05-11T15:53:00,667 copying build/lib/lm_eval/tasks/toxigen/toxigen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toxigen 2026-05-11T15:53:00,669 copying build/lib/lm_eval/tasks/toxigen/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toxigen 2026-05-11T15:53:00,671 copying build/lib/lm_eval/tasks/toxigen/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/toxigen 2026-05-11T15:53:00,673 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/icelandic_winogrande 2026-05-11T15:53:00,674 copying build/lib/lm_eval/tasks/icelandic_winogrande/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/icelandic_winogrande 2026-05-11T15:53:00,676 copying build/lib/lm_eval/tasks/icelandic_winogrande/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/icelandic_winogrande 2026-05-11T15:53:00,678 copying build/lib/lm_eval/tasks/icelandic_winogrande/preprocess_winogrande.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/icelandic_winogrande 2026-05-11T15:53:00,681 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darijahellaswag 2026-05-11T15:53:00,682 copying build/lib/lm_eval/tasks/darijahellaswag/darijahellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijahellaswag 2026-05-11T15:53:00,684 copying build/lib/lm_eval/tasks/darijahellaswag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijahellaswag 2026-05-11T15:53:00,686 copying build/lib/lm_eval/tasks/darijahellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darijahellaswag 2026-05-11T15:53:00,689 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,690 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,692 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,694 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,696 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,698 copying build/lib/lm_eval/tasks/gsm8k_platinum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,700 copying build/lib/lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-self-consistency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm8k_platinum 2026-05-11T15:53:00,702 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa 2026-05-11T15:53:00,703 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T15:53:00,705 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T15:53:00,707 copying build/lib/lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T15:53:00,709 copying build/lib/lm_eval/tasks/truthfulqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T15:53:00,710 copying build/lib/lm_eval/tasks/truthfulqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa 2026-05-11T15:53:00,713 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbq 2026-05-11T15:53:00,714 copying build/lib/lm_eval/tasks/bbq/bbq_generate_ambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,716 copying build/lib/lm_eval/tasks/bbq/bbq_multiple_choice_ambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,718 copying build/lib/lm_eval/tasks/bbq/bbq_multiple_choice_disambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,720 copying build/lib/lm_eval/tasks/bbq/bbq_generate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,722 copying build/lib/lm_eval/tasks/bbq/bbq_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,723 copying build/lib/lm_eval/tasks/bbq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,725 copying build/lib/lm_eval/tasks/bbq/bbq_generate_disambig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,727 copying build/lib/lm_eval/tasks/bbq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbq 2026-05-11T15:53:00,730 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench 2026-05-11T15:53:00,731 copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_a_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T15:53:00,736 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,737 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/timedial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,739 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,741 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,743 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,745 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,747 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,749 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,751 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,753 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,754 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,756 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,758 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,759 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,761 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,763 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,764 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,766 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,768 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,769 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,771 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,773 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,774 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,776 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,778 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,780 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,781 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,783 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,785 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,786 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,788 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,790 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,792 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,793 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,795 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,797 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,798 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,800 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,802 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,804 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,805 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,807 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,809 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,810 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,812 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,814 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,816 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,818 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,820 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,822 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,824 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,826 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,827 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,830 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,831 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,833 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,835 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,837 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,839 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,842 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,844 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,846 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,849 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,851 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,853 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,855 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,857 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,859 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,861 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,863 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,865 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,867 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,868 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,870 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,872 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,874 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,875 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,877 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,879 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,881 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,882 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,884 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,886 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,888 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,890 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,892 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,893 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,895 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,897 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,899 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kannada.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,900 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,902 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,904 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,906 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,907 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,909 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,911 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,912 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,914 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,916 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,918 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,920 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,922 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,923 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,925 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,927 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,928 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,930 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,932 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,933 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,935 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/social_support.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,937 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,939 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,941 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,942 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,944 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,946 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,948 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,950 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,951 copying build/lib/lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/multiple_choice 2026-05-11T15:53:00,957 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,958 copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,960 copying build/lib/lm_eval/tasks/bigbench/generate_until/timedial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,962 copying build/lib/lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,963 copying build/lib/lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,965 copying build/lib/lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,966 copying build/lib/lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,968 copying build/lib/lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,970 copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,971 copying build/lib/lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,973 copying build/lib/lm_eval/tasks/bigbench/generate_until/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,975 copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,977 copying build/lib/lm_eval/tasks/bigbench/generate_until/fact_checker.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,979 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,980 copying build/lib/lm_eval/tasks/bigbench/generate_until/tense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,982 copying build/lib/lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,984 copying build/lib/lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,985 copying build/lib/lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,987 copying build/lib/lm_eval/tasks/bigbench/generate_until/gem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,989 copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,991 copying build/lib/lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,993 copying build/lib/lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,994 copying build/lib/lm_eval/tasks/bigbench/generate_until/physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,996 copying build/lib/lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:00,998 copying build/lib/lm_eval/tasks/bigbench/generate_until/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,000 copying build/lib/lm_eval/tasks/bigbench/generate_until/code_line_description.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,002 copying build/lib/lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,003 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,005 copying build/lib/lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,007 copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,009 copying build/lib/lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,010 copying build/lib/lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,012 copying build/lib/lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,014 copying build/lib/lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,015 copying build/lib/lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,017 copying build/lib/lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,019 copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,021 copying build/lib/lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,022 copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,024 copying build/lib/lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,026 copying build/lib/lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,027 copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,029 copying build/lib/lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,031 copying build/lib/lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,033 copying build/lib/lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,035 copying build/lib/lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,036 copying build/lib/lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,038 copying build/lib/lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,040 copying build/lib/lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,041 copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,043 copying build/lib/lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,045 copying build/lib/lm_eval/tasks/bigbench/generate_until/operators.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,046 copying build/lib/lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,048 copying build/lib/lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,050 copying build/lib/lm_eval/tasks/bigbench/generate_until/codenames.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,051 copying build/lib/lm_eval/tasks/bigbench/generate_until/implicatures.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,053 copying build/lib/lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,055 copying build/lib/lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,057 copying build/lib/lm_eval/tasks/bigbench/generate_until/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,059 copying build/lib/lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,061 copying build/lib/lm_eval/tasks/bigbench/generate_until/rephrase.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,062 copying build/lib/lm_eval/tasks/bigbench/generate_until/multiemo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,064 copying build/lib/lm_eval/tasks/bigbench/generate_until/physics_questions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,066 copying build/lib/lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,068 copying build/lib/lm_eval/tasks/bigbench/generate_until/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,069 copying build/lib/lm_eval/tasks/bigbench/generate_until/topical_chat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,071 copying build/lib/lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,073 copying build/lib/lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,075 copying build/lib/lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,077 copying build/lib/lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,079 copying build/lib/lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,080 copying build/lib/lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,082 copying build/lib/lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,084 copying build/lib/lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,085 copying build/lib/lm_eval/tasks/bigbench/generate_until/question_selection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,087 copying build/lib/lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,089 copying build/lib/lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,090 copying build/lib/lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,092 copying build/lib/lm_eval/tasks/bigbench/generate_until/winowhy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,094 copying build/lib/lm_eval/tasks/bigbench/generate_until/cryptonite.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,096 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,097 copying build/lib/lm_eval/tasks/bigbench/generate_until/color.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,099 copying build/lib/lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,101 copying build/lib/lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,103 copying build/lib/lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,104 copying build/lib/lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,106 copying build/lib/lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,108 copying build/lib/lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,109 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_args.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,111 copying build/lib/lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,113 copying build/lib/lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,115 copying build/lib/lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,116 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,118 copying build/lib/lm_eval/tasks/bigbench/generate_until/strange_stories.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,120 copying build/lib/lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,121 copying build/lib/lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,123 copying build/lib/lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,125 copying build/lib/lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,127 copying build/lib/lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,128 copying build/lib/lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,130 copying build/lib/lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,132 copying build/lib/lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,133 copying build/lib/lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,135 copying build/lib/lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,137 copying build/lib/lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,139 copying build/lib/lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,141 copying build/lib/lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,142 copying build/lib/lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,144 copying build/lib/lm_eval/tasks/bigbench/generate_until/arithmetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,146 copying build/lib/lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,148 copying build/lib/lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,150 copying build/lib/lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,152 copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,154 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,155 copying build/lib/lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,157 copying build/lib/lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,159 copying build/lib/lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,160 copying build/lib/lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,162 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,164 copying build/lib/lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,166 copying build/lib/lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,167 copying build/lib/lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,169 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,171 copying build/lib/lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,173 copying build/lib/lm_eval/tasks/bigbench/generate_until/anachronisms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,178 copying build/lib/lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,180 copying build/lib/lm_eval/tasks/bigbench/generate_until/social_iqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,182 copying build/lib/lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,184 copying build/lib/lm_eval/tasks/bigbench/generate_until/kannada.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,185 copying build/lib/lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,187 copying build/lib/lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,189 copying build/lib/lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,191 copying build/lib/lm_eval/tasks/bigbench/generate_until/strategyqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,192 copying build/lib/lm_eval/tasks/bigbench/generate_until/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,194 copying build/lib/lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,196 copying build/lib/lm_eval/tasks/bigbench/generate_until/irony_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,198 copying build/lib/lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,199 copying build/lib/lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,201 copying build/lib/lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,203 copying build/lib/lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,204 copying build/lib/lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,206 copying build/lib/lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,208 copying build/lib/lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,210 copying build/lib/lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,212 copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,213 copying build/lib/lm_eval/tasks/bigbench/generate_until/language_games.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,215 copying build/lib/lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,217 copying build/lib/lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,219 copying build/lib/lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,221 copying build/lib/lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,223 copying build/lib/lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,225 copying build/lib/lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,226 copying build/lib/lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,228 copying build/lib/lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,230 copying build/lib/lm_eval/tasks/bigbench/generate_until/language_identification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,232 copying build/lib/lm_eval/tasks/bigbench/generate_until/social_support.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,234 copying build/lib/lm_eval/tasks/bigbench/generate_until/list_functions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,235 copying build/lib/lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,237 copying build/lib/lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,239 copying build/lib/lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,242 copying build/lib/lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,243 copying build/lib/lm_eval/tasks/bigbench/generate_until/crass_ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,245 copying build/lib/lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,247 copying build/lib/lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,249 copying build/lib/lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,251 copying build/lib/lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,252 copying build/lib/lm_eval/tasks/bigbench/generate_until/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,254 copying build/lib/lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench/generate_until 2026-05-11T15:53:01,256 copying build/lib/lm_eval/tasks/bigbench/generate_tasks.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T15:53:01,258 copying build/lib/lm_eval/tasks/bigbench/multiple_choice_template_b_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T15:53:01,260 copying build/lib/lm_eval/tasks/bigbench/push_bigbench_dataset.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T15:53:01,262 copying build/lib/lm_eval/tasks/bigbench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T15:53:01,264 copying build/lib/lm_eval/tasks/bigbench/generate_until_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bigbench 2026-05-11T15:53:01,266 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T15:53:01,268 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:53:01,269 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_leaderboard_arabic_mt_race.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:53:01,271 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_mt_race.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:53:01,273 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race 2026-05-11T15:53:01,276 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:53:01,277 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_mt_arc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:53:01,279 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_leaderboard_arabic_mt_arc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:53:01,281 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy 2026-05-11T15:53:01,283 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_complete.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T15:53:01,286 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,287 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,289 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,291 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,293 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,295 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,296 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,298 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,300 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,302 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,304 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,305 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,307 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,309 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,311 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,313 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,315 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,316 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,318 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,320 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,322 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,324 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,325 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,327 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,329 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,331 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,333 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,335 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,337 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,338 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,340 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,342 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,344 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,345 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,347 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,349 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,351 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,352 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,354 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,356 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,359 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,361 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,363 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,365 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,368 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,370 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,372 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,374 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,377 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,379 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,381 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,383 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,385 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,388 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,391 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,396 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,402 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,406 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,408 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,411 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu 2026-05-11T15:53:01,415 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:53:01,417 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_mt_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:53:01,421 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_leaderboard_arabic_mt_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:53:01,424 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu 2026-05-11T15:53:01,428 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:53:01,430 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_mt_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:53:01,433 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_leaderboard_arabic_mt_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:53:01,437 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge 2026-05-11T15:53:01,441 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:53:01,443 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_leaderboard_arabic_mt_toxigen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:53:01,447 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_mt_toxigen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:53:01,450 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen 2026-05-11T15:53:01,453 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:53:01,454 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_leaderboard_arabic_mt_boolq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:53:01,457 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_mt_boolq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:53:01,459 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq 2026-05-11T15:53:01,462 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:53:01,463 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_mt_sciq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:53:01,467 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_leaderboard_arabic_mt_sciq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:53:01,471 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq 2026-05-11T15:53:01,476 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,477 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ceremony.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,481 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mauritania.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,484 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Jordan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,486 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_United_Arab_Emirates.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,489 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,492 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Comoros.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,495 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,498 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromIslam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,502 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,506 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Lebanon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,510 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,512 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,514 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Art.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,516 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Palestine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,519 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Calligraphy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,521 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Tunisia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,523 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,525 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arab_Empire.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,527 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Iraq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,529 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,532 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Somalia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,534 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Food.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,536 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Libya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,538 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromAncientEgypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,540 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Oman.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,542 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromGreece.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,545 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mesopotamia_civilization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,547 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_communication.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,549 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Bahrain.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,551 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islamic_law_system.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,553 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Language_Origin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,555 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromByzantium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,558 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_Education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,560 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_branches_and_schools.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,562 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromChina.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,564 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromPersia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,566 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_entertainment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,568 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_daily_life.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,570 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_History.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,573 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Saudi_Arabia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,575 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromRome.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,577 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Wedding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,579 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,581 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,584 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Qatar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,586 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Sudan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,588 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Ancient_Egypt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,590 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,591 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Syria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,593 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Morocco.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,595 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_computer_and_phone.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,596 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Yemen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,598 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Clothing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,600 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,602 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Egypt_modern.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,603 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Funeral.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,605 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,607 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ornament.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,608 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Kuwait.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,610 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Algeria.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca 2026-05-11T15:53:01,613 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:53:01,614 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_mt_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:53:01,616 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_leaderboard_arabic_mt_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:53:01,617 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag 2026-05-11T15:53:01,619 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete 2026-05-11T15:53:01,622 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:53:01,623 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_mt_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:53:01,625 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_leaderboard_arabic_mt_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:53:01,626 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa 2026-05-11T15:53:01,629 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:53:01,630 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_leaderboard_arabic_mt_openbook_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:53:01,632 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_mt_openbook_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:53:01,633 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa 2026-05-11T15:53:01,636 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:53:01,637 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_exams.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:53:01,638 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_leaderboard_arabic_exams.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:53:01,640 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams 2026-05-11T15:53:01,643 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:53:01,644 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_mt_copa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:53:01,645 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_leaderboard_arabic_mt_copa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:53:01,647 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa 2026-05-11T15:53:01,649 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,651 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,653 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_dialects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,655 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_sentiment_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,656 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_mcq_exams_test_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,658 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,660 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,662 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,664 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,666 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,668 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,669 copying build/lib/lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa 2026-05-11T15:53:01,672 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medqa 2026-05-11T15:53:01,673 copying build/lib/lm_eval/tasks/medqa/medqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medqa 2026-05-11T15:53:01,674 copying build/lib/lm_eval/tasks/medqa/preprocess_medqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medqa 2026-05-11T15:53:01,677 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pubmedqa 2026-05-11T15:53:01,678 copying build/lib/lm_eval/tasks/pubmedqa/pubmedqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pubmedqa 2026-05-11T15:53:01,680 copying build/lib/lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pubmedqa 2026-05-11T15:53:01,681 copying build/lib/lm_eval/tasks/pubmedqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pubmedqa 2026-05-11T15:53:01,684 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/scrolls 2026-05-11T15:53:01,685 copying build/lib/lm_eval/tasks/scrolls/scrolls_govreport.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,687 copying build/lib/lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,688 copying build/lib/lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,690 copying build/lib/lm_eval/tasks/scrolls/scrolls_qmsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,692 copying build/lib/lm_eval/tasks/scrolls/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,694 copying build/lib/lm_eval/tasks/scrolls/scrolls_qasper.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,696 copying build/lib/lm_eval/tasks/scrolls/scrolls_contractnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,697 copying build/lib/lm_eval/tasks/scrolls/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,699 copying build/lib/lm_eval/tasks/scrolls/scrolls_quality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/scrolls 2026-05-11T15:53:01,702 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mlqa 2026-05-11T15:53:01,703 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,705 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,707 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,709 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,711 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,712 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,714 copying build/lib/lm_eval/tasks/mlqa/mlqa_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,716 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,717 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,719 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,721 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,723 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,724 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,726 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,728 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,730 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,732 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,734 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,736 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,738 copying build/lib/lm_eval/tasks/mlqa/generate_tasks.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,740 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,742 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,743 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,745 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,746 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,748 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,750 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,752 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,753 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,755 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,757 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,758 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,760 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,762 copying build/lib/lm_eval/tasks/mlqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,764 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,766 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,767 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,769 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,771 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,773 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,774 copying build/lib/lm_eval/tasks/mlqa/mlqa_es_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,776 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,778 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,780 copying build/lib/lm_eval/tasks/mlqa/mlqa_zh_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,781 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,783 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,785 copying build/lib/lm_eval/tasks/mlqa/mlqa_en_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,787 copying build/lib/lm_eval/tasks/mlqa/mlqa_ar_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,789 copying build/lib/lm_eval/tasks/mlqa/mlqa_hi_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,790 copying build/lib/lm_eval/tasks/mlqa/mlqa_de_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,792 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,794 copying build/lib/lm_eval/tasks/mlqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,796 copying build/lib/lm_eval/tasks/mlqa/mlqa_vi_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mlqa 2026-05-11T15:53:01,798 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval 2026-05-11T15:53:01,800 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba 2026-05-11T15:53:01,801 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:53:01,803 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:53:01,804 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:53:01,806 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:53:01,808 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob 2026-05-11T15:53:01,810 copying build/lib/lm_eval/tasks/noreval/tatoeba/_tatoeba_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba 2026-05-11T15:53:01,812 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:53:01,813 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:53:01,815 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:53:01,817 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:53:01,818 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng 2026-05-11T15:53:01,821 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:53:01,822 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:53:01,824 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:53:01,825 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:53:01,827 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng 2026-05-11T15:53:01,830 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:53:01,831 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:53:01,833 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:53:01,834 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:53:01,836 copying build/lib/lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno 2026-05-11T15:53:01,838 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noropenbookqa 2026-05-11T15:53:01,840 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:53:01,841 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:53:01,843 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:53:01,845 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:53:01,847 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:53:01,849 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nob 2026-05-11T15:53:01,851 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:53:01,852 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:53:01,854 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:53:01,856 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:53:01,857 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:53:01,859 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa/nno 2026-05-11T15:53:01,861 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/_noropenbookqa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa 2026-05-11T15:53:01,863 copying build/lib/lm_eval/tasks/noreval/noropenbookqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noropenbookqa 2026-05-11T15:53:01,865 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nrk_quiz_qa 2026-05-11T15:53:01,866 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,867 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,869 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,871 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,873 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,875 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,876 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nob 2026-05-11T15:53:01,879 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,880 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,881 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,883 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,885 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,886 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,888 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa/nno 2026-05-11T15:53:01,890 copying build/lib/lm_eval/tasks/noreval/nrk_quiz_qa/_nrk_quiz_qa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nrk_quiz_qa 2026-05-11T15:53:01,892 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norec 2026-05-11T15:53:01,893 copying build/lib/lm_eval/tasks/noreval/norec/_norec_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec 2026-05-11T15:53:01,895 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:53:01,896 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:53:01,899 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:53:01,900 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:53:01,902 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:53:01,904 copying build/lib/lm_eval/tasks/noreval/norec/norec_document/norec_document_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_document 2026-05-11T15:53:01,906 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:53:01,907 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:53:01,909 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:53:01,911 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:53:01,912 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:53:01,914 copying build/lib/lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec/norec_sentence 2026-05-11T15:53:01,916 copying build/lib/lm_eval/tasks/noreval/norec/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norec 2026-05-11T15:53:01,918 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noridiom 2026-05-11T15:53:01,920 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:53:01,921 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:53:01,923 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:53:01,924 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:53:01,926 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:53:01,928 copying build/lib/lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nob 2026-05-11T15:53:01,930 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:53:01,931 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:53:01,933 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:53:01,935 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:53:01,936 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:53:01,938 copying build/lib/lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom/nno 2026-05-11T15:53:01,940 copying build/lib/lm_eval/tasks/noreval/noridiom/_noridiom_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom 2026-05-11T15:53:01,942 copying build/lib/lm_eval/tasks/noreval/noridiom/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/noridiom 2026-05-11T15:53:01,944 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa 2026-05-11T15:53:01,946 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice 2026-05-11T15:53:01,947 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,948 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,950 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,952 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,954 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,956 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,957 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob 2026-05-11T15:53:01,959 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/_nortruthfulqa_mc_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice 2026-05-11T15:53:01,962 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,962 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,964 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,966 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,968 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,970 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,971 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno 2026-05-11T15:53:01,974 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T15:53:01,975 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:53:01,976 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:53:01,978 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:53:01,980 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:53:01,982 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:53:01,984 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nob 2026-05-11T15:53:01,986 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:53:01,987 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:53:01,989 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:53:01,991 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:53:01,992 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:53:01,994 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation/nno 2026-05-11T15:53:01,996 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/_nortruthfulqa_gen_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T15:53:01,998 copying build/lib/lm_eval/tasks/noreval/nortruthfulqa/generation/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/nortruthfulqa/generation 2026-05-11T15:53:02,001 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsumm 2026-05-11T15:53:02,002 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,003 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,005 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,007 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,009 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,011 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,012 copying build/lib/lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nob 2026-05-11T15:53:02,014 copying build/lib/lm_eval/tasks/noreval/norsumm/_norsumm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm 2026-05-11T15:53:02,017 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,018 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,020 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,021 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,023 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,025 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,027 copying build/lib/lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm/nno 2026-05-11T15:53:02,029 copying build/lib/lm_eval/tasks/noreval/norsumm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsumm 2026-05-11T15:53:02,032 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,033 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,035 copying build/lib/lm_eval/tasks/noreval/norquad/_norquad_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,037 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,039 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,041 copying build/lib/lm_eval/tasks/noreval/norquad/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,043 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,045 copying build/lib/lm_eval/tasks/noreval/norquad/norquad_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norquad 2026-05-11T15:53:02,047 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T15:53:02,048 copying build/lib/lm_eval/tasks/noreval/norsummarize-instruct/norsummarize_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norsummarize-instruct 2026-05-11T15:53:02,050 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T15:53:02,051 copying build/lib/lm_eval/tasks/noreval/norrewrite-instruct/norrewrite_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norrewrite-instruct 2026-05-11T15:53:02,054 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norcommonsenseqa 2026-05-11T15:53:02,055 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:53:02,056 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:53:02,058 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:53:02,060 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:53:02,062 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:53:02,063 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nob 2026-05-11T15:53:02,065 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/_norcommonsenseqa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa 2026-05-11T15:53:02,067 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:53:02,068 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:53:02,070 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:53:02,072 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:53:02,073 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:53:02,075 copying build/lib/lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norcommonsenseqa/nno 2026-05-11T15:53:02,078 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/ncb 2026-05-11T15:53:02,079 copying build/lib/lm_eval/tasks/noreval/ncb/ncb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ncb 2026-05-11T15:53:02,081 copying build/lib/lm_eval/tasks/noreval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval 2026-05-11T15:53:02,083 copying build/lib/lm_eval/tasks/noreval/noreval.jpg -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval 2026-05-11T15:53:02,095 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,097 copying build/lib/lm_eval/tasks/noreval/norbelebele/_norbelebele_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,099 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,100 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,102 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,104 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,106 copying build/lib/lm_eval/tasks/noreval/norbelebele/norbelebele_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/norbelebele 2026-05-11T15:53:02,109 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,110 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,111 copying build/lib/lm_eval/tasks/noreval/ask_gec/errant.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,113 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,115 copying build/lib/lm_eval/tasks/noreval/ask_gec/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,117 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,118 copying build/lib/lm_eval/tasks/noreval/ask_gec/_ask_gec_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,120 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p0.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,122 copying build/lib/lm_eval/tasks/noreval/ask_gec/ask_gec_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/noreval/ask_gec 2026-05-11T15:53:02,124 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_trivia 2026-05-11T15:53:02,125 copying build/lib/lm_eval/tasks/eus_trivia/eus_trivia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_trivia 2026-05-11T15:53:02,127 copying build/lib/lm_eval/tasks/eus_trivia/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_trivia 2026-05-11T15:53:02,129 copying build/lib/lm_eval/tasks/eus_trivia/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_trivia 2026-05-11T15:53:02,131 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ulqa 2026-05-11T15:53:02,132 copying build/lib/lm_eval/tasks/ulqa/celep1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,134 copying build/lib/lm_eval/tasks/ulqa/celep2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,136 copying build/lib/lm_eval/tasks/ulqa/lambada_uyghur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,138 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,139 copying build/lib/lm_eval/tasks/ulqa/ulut/wsm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,141 copying build/lib/lm_eval/tasks/ulqa/ulut/wag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,143 copying build/lib/lm_eval/tasks/ulqa/ulut/wum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,145 copying build/lib/lm_eval/tasks/ulqa/ulut/ulut.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,147 copying build/lib/lm_eval/tasks/ulqa/ulut/nug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,148 copying build/lib/lm_eval/tasks/ulqa/ulut/wub.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa/ulut 2026-05-11T15:53:02,150 copying build/lib/lm_eval/tasks/ulqa/uleval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,152 copying build/lib/lm_eval/tasks/ulqa/ulqa_.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,153 copying build/lib/lm_eval/tasks/ulqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,155 copying build/lib/lm_eval/tasks/ulqa/ulqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ulqa 2026-05-11T15:53:02,157 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fda 2026-05-11T15:53:02,159 copying build/lib/lm_eval/tasks/fda/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fda 2026-05-11T15:53:02,161 copying build/lib/lm_eval/tasks/fda/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fda 2026-05-11T15:53:02,163 copying build/lib/lm_eval/tasks/fda/fda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fda 2026-05-11T15:53:02,165 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/coqa 2026-05-11T15:53:02,166 copying build/lib/lm_eval/tasks/coqa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/coqa 2026-05-11T15:53:02,168 copying build/lib/lm_eval/tasks/coqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/coqa 2026-05-11T15:53:02,169 copying build/lib/lm_eval/tasks/coqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/coqa 2026-05-11T15:53:02,172 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli_eu 2026-05-11T15:53:02,173 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T15:53:02,175 copying build/lib/lm_eval/tasks/xnli_eu/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T15:53:02,177 copying build/lib/lm_eval/tasks/xnli_eu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T15:53:02,179 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu_native.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T15:53:02,180 copying build/lib/lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli_eu 2026-05-11T15:53:02,183 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medmcqa 2026-05-11T15:53:02,184 copying build/lib/lm_eval/tasks/medmcqa/medmcqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medmcqa 2026-05-11T15:53:02,186 copying build/lib/lm_eval/tasks/medmcqa/utils_medmcqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medmcqa 2026-05-11T15:53:02,188 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ruler 2026-05-11T15:53:02,189 copying build/lib/lm_eval/tasks/ruler/fwe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,191 copying build/lib/lm_eval/tasks/ruler/prepare_niah.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,194 copying build/lib/lm_eval/tasks/ruler/niah_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,196 copying build/lib/lm_eval/tasks/ruler/niah_multikey_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,198 copying build/lib/lm_eval/tasks/ruler/qa_squad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,200 copying build/lib/lm_eval/tasks/ruler/niah_multivalue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,202 copying build/lib/lm_eval/tasks/ruler/qa_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,204 copying build/lib/lm_eval/tasks/ruler/vt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,205 copying build/lib/lm_eval/tasks/ruler/ruler.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,207 copying build/lib/lm_eval/tasks/ruler/niah_multikey_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,209 copying build/lib/lm_eval/tasks/ruler/essays.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,211 copying build/lib/lm_eval/tasks/ruler/niah_single_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,213 copying build/lib/lm_eval/tasks/ruler/niah_multikey_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,215 copying build/lib/lm_eval/tasks/ruler/vt_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,217 copying build/lib/lm_eval/tasks/ruler/niah_multiquery.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,218 copying build/lib/lm_eval/tasks/ruler/niah_single_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,220 copying build/lib/lm_eval/tasks/ruler/qa_hotpot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,222 copying build/lib/lm_eval/tasks/ruler/niah_single_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,223 copying build/lib/lm_eval/tasks/ruler/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,225 copying build/lib/lm_eval/tasks/ruler/fwe_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,227 copying build/lib/lm_eval/tasks/ruler/common_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,229 copying build/lib/lm_eval/tasks/ruler/cwe_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,231 copying build/lib/lm_eval/tasks/ruler/cwe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ruler 2026-05-11T15:53:02,234 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,235 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,237 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,239 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,241 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jaqket_v2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,242 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jsquad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,245 copying build/lib/lm_eval/tasks/japanese_leaderboard/requirements.txt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,246 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,248 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,250 copying build/lib/lm_eval/tasks/japanese_leaderboard/_ja_leaderboard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,252 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,253 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,255 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,257 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_marc_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,259 copying build/lib/lm_eval/tasks/japanese_leaderboard/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,261 copying build/lib/lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/japanese_leaderboard 2026-05-11T15:53:02,264 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals 2026-05-11T15:53:02,265 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:53:02,266 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:53:02,268 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:53:02,270 copying build/lib/lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/sycophancy 2026-05-11T15:53:02,274 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,275 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,277 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,279 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,281 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,282 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,284 copying build/lib/lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,286 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,288 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,289 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,291 copying build/lib/lm_eval/tasks/model_written_evals/persona/openness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,293 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,295 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,297 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,299 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,300 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,302 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,304 copying build/lib/lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,306 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,307 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,309 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,311 copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,312 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,314 copying build/lib/lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,316 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,318 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,320 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,322 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,323 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,325 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,327 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,329 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,331 copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,332 copying build/lib/lm_eval/tasks/model_written_evals/persona/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,334 copying build/lib/lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,336 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,338 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,340 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,341 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,343 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,344 copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,346 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,348 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,350 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,351 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,353 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,355 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,357 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,358 copying build/lib/lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,360 copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,362 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,364 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,365 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,367 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,369 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,371 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,372 copying build/lib/lm_eval/tasks/model_written_evals/persona/neuroticism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,374 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,376 copying build/lib/lm_eval/tasks/model_written_evals/persona/agreeableness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,378 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,380 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,381 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,383 copying build/lib/lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,385 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,387 copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,389 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,391 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,393 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,394 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,396 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,398 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,400 copying build/lib/lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,401 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,403 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,406 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,408 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,409 copying build/lib/lm_eval/tasks/model_written_evals/persona/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,411 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,413 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,414 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,416 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,418 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,420 copying build/lib/lm_eval/tasks/model_written_evals/persona/self-replication.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,421 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,423 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,425 copying build/lib/lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,427 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,429 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,431 copying build/lib/lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,433 copying build/lib/lm_eval/tasks/model_written_evals/persona/narcissism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,435 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,437 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,438 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,440 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,442 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,444 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,446 copying build/lib/lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,447 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,449 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,451 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,453 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,454 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,456 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,458 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,460 copying build/lib/lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,462 copying build/lib/lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,464 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,466 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,467 copying build/lib/lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,469 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,471 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,473 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,475 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,477 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,479 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,480 copying build/lib/lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,482 copying build/lib/lm_eval/tasks/model_written_evals/persona/risk-averse.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,484 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,485 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,487 copying build/lib/lm_eval/tasks/model_written_evals/persona/extraversion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,489 copying build/lib/lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,491 copying build/lib/lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,493 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,495 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,496 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,499 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,500 copying build/lib/lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,502 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,504 copying build/lib/lm_eval/tasks/model_written_evals/persona/psychopathy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,506 copying build/lib/lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,508 copying build/lib/lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,510 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,512 copying build/lib/lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,514 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,515 copying build/lib/lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,517 copying build/lib/lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,519 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,521 copying build/lib/lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/persona 2026-05-11T15:53:02,523 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T15:53:02,524 copying build/lib/lm_eval/tasks/model_written_evals/winogenerated/winogenerated -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/winogenerated 2026-05-11T15:53:02,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,528 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,530 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,532 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,533 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,535 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,537 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,539 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,541 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,542 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,544 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,546 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,548 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,550 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,552 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,553 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,555 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,557 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,559 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,561 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,563 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,564 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,566 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,568 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,570 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,571 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,573 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,575 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,577 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,579 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,581 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,583 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,584 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,586 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,588 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,590 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,592 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,593 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,595 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,597 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,599 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,600 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,602 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,604 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,606 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,608 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,609 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,611 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,613 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,615 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,617 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,619 copying build/lib/lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/model_written_evals/advanced_ai_risk 2026-05-11T15:53:02,621 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wikitext 2026-05-11T15:53:02,622 copying build/lib/lm_eval/tasks/wikitext/wikitext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wikitext 2026-05-11T15:53:02,624 copying build/lib/lm_eval/tasks/wikitext/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wikitext 2026-05-11T15:53:02,626 copying build/lib/lm_eval/tasks/wikitext/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wikitext 2026-05-11T15:53:02,628 copying build/lib/lm_eval/tasks/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T15:53:02,631 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc 2026-05-11T15:53:02,632 copying build/lib/lm_eval/tasks/arc/arc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T15:53:02,633 copying build/lib/lm_eval/tasks/arc/test_arc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T15:53:02,635 copying build/lib/lm_eval/tasks/arc/arc_challenge_chat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T15:53:02,637 copying build/lib/lm_eval/tasks/arc/arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T15:53:02,638 copying build/lib/lm_eval/tasks/arc/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T15:53:02,640 copying build/lib/lm_eval/tasks/arc/groups.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc 2026-05-11T15:53:02,642 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/swag 2026-05-11T15:53:02,643 copying build/lib/lm_eval/tasks/swag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swag 2026-05-11T15:53:02,645 copying build/lib/lm_eval/tasks/swag/swag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/swag 2026-05-11T15:53:02,648 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/med_text_classification 2026-05-11T15:53:02,649 copying build/lib/lm_eval/tasks/med_text_classification/med_text_classification_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_text_classification 2026-05-11T15:53:02,651 copying build/lib/lm_eval/tasks/med_text_classification/med_text_classification_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_text_classification 2026-05-11T15:53:02,653 copying build/lib/lm_eval/tasks/med_text_classification/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_text_classification 2026-05-11T15:53:02,655 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh 2026-05-11T15:53:02,657 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,659 copying build/lib/lm_eval/tasks/bbh/fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,661 copying build/lib/lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,663 copying build/lib/lm_eval/tasks/bbh/fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,665 copying build/lib/lm_eval/tasks/bbh/fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,666 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,669 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,670 copying build/lib/lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,672 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,674 copying build/lib/lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,676 copying build/lib/lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,678 copying build/lib/lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,680 copying build/lib/lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,682 copying build/lib/lm_eval/tasks/bbh/fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,683 copying build/lib/lm_eval/tasks/bbh/fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,685 copying build/lib/lm_eval/tasks/bbh/fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,687 copying build/lib/lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,689 copying build/lib/lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,690 copying build/lib/lm_eval/tasks/bbh/fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,692 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,694 copying build/lib/lm_eval/tasks/bbh/fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,695 copying build/lib/lm_eval/tasks/bbh/fewshot/_bbh_fewshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,697 copying build/lib/lm_eval/tasks/bbh/fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,699 copying build/lib/lm_eval/tasks/bbh/fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,701 copying build/lib/lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,703 copying build/lib/lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,704 copying build/lib/lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,706 copying build/lib/lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,708 copying build/lib/lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,710 copying build/lib/lm_eval/tasks/bbh/fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/fewshot 2026-05-11T15:53:02,713 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,714 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,716 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,718 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,720 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,722 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,724 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,726 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_bbh_cot_fewshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,728 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,730 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,732 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,734 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,736 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,738 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,739 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,741 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,743 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,745 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,747 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,749 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/_bbh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,751 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,753 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,755 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,757 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,759 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,761 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,763 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,765 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,767 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,769 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,771 copying build/lib/lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_fewshot 2026-05-11T15:53:02,773 copying build/lib/lm_eval/tasks/bbh/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh 2026-05-11T15:53:02,775 copying build/lib/lm_eval/tasks/bbh/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh 2026-05-11T15:53:02,777 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,778 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,780 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,782 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,784 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,786 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,787 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,789 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,791 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,792 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,794 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,796 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,798 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,800 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,802 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,803 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,805 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,807 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,809 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/_bbh_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,811 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,812 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,814 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,816 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,818 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,820 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,822 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,824 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,825 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,828 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,829 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,831 copying build/lib/lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/cot_zeroshot 2026-05-11T15:53:02,834 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,835 copying build/lib/lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,837 copying build/lib/lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,838 copying build/lib/lm_eval/tasks/bbh/zeroshot/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,840 copying build/lib/lm_eval/tasks/bbh/zeroshot/word_sorting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,842 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,844 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,846 copying build/lib/lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,848 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,850 copying build/lib/lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,851 copying build/lib/lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,853 copying build/lib/lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,855 copying build/lib/lm_eval/tasks/bbh/zeroshot/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,857 copying build/lib/lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,859 copying build/lib/lm_eval/tasks/bbh/zeroshot/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,860 copying build/lib/lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,862 copying build/lib/lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,864 copying build/lib/lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,866 copying build/lib/lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,868 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,870 copying build/lib/lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,871 copying build/lib/lm_eval/tasks/bbh/zeroshot/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,873 copying build/lib/lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,875 copying build/lib/lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,876 copying build/lib/lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,878 copying build/lib/lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,880 copying build/lib/lm_eval/tasks/bbh/zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,882 copying build/lib/lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,884 copying build/lib/lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,886 copying build/lib/lm_eval/tasks/bbh/zeroshot/_bbh_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,888 copying build/lib/lm_eval/tasks/bbh/zeroshot/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bbh/zeroshot 2026-05-11T15:53:02,890 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/csatqa 2026-05-11T15:53:02,891 copying build/lib/lm_eval/tasks/csatqa/csatqa_li.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,893 copying build/lib/lm_eval/tasks/csatqa/_csatqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,895 copying build/lib/lm_eval/tasks/csatqa/_default_csatqa_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,896 copying build/lib/lm_eval/tasks/csatqa/csatqa_wr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,898 copying build/lib/lm_eval/tasks/csatqa/csatqa_gr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,900 copying build/lib/lm_eval/tasks/csatqa/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,902 copying build/lib/lm_eval/tasks/csatqa/csatqa_rch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,903 copying build/lib/lm_eval/tasks/csatqa/csatqa_rcss.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,905 copying build/lib/lm_eval/tasks/csatqa/csatqa_rcs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,907 copying build/lib/lm_eval/tasks/csatqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/csatqa 2026-05-11T15:53:02,909 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xnli 2026-05-11T15:53:02,910 copying build/lib/lm_eval/tasks/xnli/xnli_bg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,912 copying build/lib/lm_eval/tasks/xnli/xnli_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,914 copying build/lib/lm_eval/tasks/xnli/xnli_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,916 copying build/lib/lm_eval/tasks/xnli/xnli_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,917 copying build/lib/lm_eval/tasks/xnli/xnli_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,919 copying build/lib/lm_eval/tasks/xnli/xnli_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,921 copying build/lib/lm_eval/tasks/xnli/xnli_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,923 copying build/lib/lm_eval/tasks/xnli/xnli_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,924 copying build/lib/lm_eval/tasks/xnli/xnli_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,926 copying build/lib/lm_eval/tasks/xnli/xnli_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,928 copying build/lib/lm_eval/tasks/xnli/xnli_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,929 copying build/lib/lm_eval/tasks/xnli/_xnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,931 copying build/lib/lm_eval/tasks/xnli/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,933 copying build/lib/lm_eval/tasks/xnli/xnli_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,935 copying build/lib/lm_eval/tasks/xnli/xnli_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,936 copying build/lib/lm_eval/tasks/xnli/xnli_ur.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,938 copying build/lib/lm_eval/tasks/xnli/xnli_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,940 copying build/lib/lm_eval/tasks/xnli/xnli_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,941 copying build/lib/lm_eval/tasks/xnli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xnli 2026-05-11T15:53:02,944 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_piqa 2026-05-11T15:53:02,947 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,949 copying build/lib/lm_eval/tasks/global_piqa/completions/nag_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,950 copying build/lib/lm_eval/tasks/global_piqa/completions/por_latn_braz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,952 copying build/lib/lm_eval/tasks/global_piqa/completions/zul_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,954 copying build/lib/lm_eval/tasks/global_piqa/completions/hin_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,956 copying build/lib/lm_eval/tasks/global_piqa/completions/hye_armn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,958 copying build/lib/lm_eval/tasks/global_piqa/completions/vie_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,959 copying build/lib/lm_eval/tasks/global_piqa/completions/bho_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,961 copying build/lib/lm_eval/tasks/global_piqa/completions/fin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,963 copying build/lib/lm_eval/tasks/global_piqa/completions/bam_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,965 copying build/lib/lm_eval/tasks/global_piqa/completions/urh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,966 copying build/lib/lm_eval/tasks/global_piqa/completions/rwr_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,968 copying build/lib/lm_eval/tasks/global_piqa/completions/spa_latn_mexi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,970 copying build/lib/lm_eval/tasks/global_piqa/completions/acq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,972 copying build/lib/lm_eval/tasks/global_piqa/completions/ita_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,974 copying build/lib/lm_eval/tasks/global_piqa/completions/ekk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,975 copying build/lib/lm_eval/tasks/global_piqa/completions/cat_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,977 copying build/lib/lm_eval/tasks/global_piqa/completions/isl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,979 copying build/lib/lm_eval/tasks/global_piqa/completions/_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,980 copying build/lib/lm_eval/tasks/global_piqa/completions/ckm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,982 copying build/lib/lm_eval/tasks/global_piqa/completions/yor_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,984 copying build/lib/lm_eval/tasks/global_piqa/completions/uzn_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,985 copying build/lib/lm_eval/tasks/global_piqa/completions/hun_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,987 copying build/lib/lm_eval/tasks/global_piqa/completions/pol_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,989 copying build/lib/lm_eval/tasks/global_piqa/completions/mni_mtei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,990 copying build/lib/lm_eval/tasks/global_piqa/completions/slk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,992 copying build/lib/lm_eval/tasks/global_piqa/completions/glg_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,994 copying build/lib/lm_eval/tasks/global_piqa/completions/bel_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,996 copying build/lib/lm_eval/tasks/global_piqa/completions/tel_telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:02,998 copying build/lib/lm_eval/tasks/global_piqa/completions/swe_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,000 copying build/lib/lm_eval/tasks/global_piqa/completions/yue_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,002 copying build/lib/lm_eval/tasks/global_piqa/completions/jpn_jpan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,003 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_pale.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,005 copying build/lib/lm_eval/tasks/global_piqa/completions/snd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,007 copying build/lib/lm_eval/tasks/global_piqa/completions/slv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,009 copying build/lib/lm_eval/tasks/global_piqa/completions/dhd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,011 copying build/lib/lm_eval/tasks/global_piqa/completions/bos_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,012 copying build/lib/lm_eval/tasks/global_piqa/completions/slv_latn_cerk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,014 copying build/lib/lm_eval/tasks/global_piqa/completions/ukr_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,016 copying build/lib/lm_eval/tasks/global_piqa/completions/guj_gujr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,018 copying build/lib/lm_eval/tasks/global_piqa/completions/tam_taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,019 copying build/lib/lm_eval/tasks/global_piqa/completions/fra_latn_fran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,021 copying build/lib/lm_eval/tasks/global_piqa/completions/mni_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,023 copying build/lib/lm_eval/tasks/global_piqa/completions/lit_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,025 copying build/lib/lm_eval/tasks/global_piqa/completions/jav_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,026 copying build/lib/lm_eval/tasks/global_piqa/completions/_global_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,029 copying build/lib/lm_eval/tasks/global_piqa/completions/tha_thai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,031 copying build/lib/lm_eval/tasks/global_piqa/completions/srp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,032 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_leba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,034 copying build/lib/lm_eval/tasks/global_piqa/completions/spa_latn_peru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,036 copying build/lib/lm_eval/tasks/global_piqa/completions/ind_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,037 copying build/lib/lm_eval/tasks/global_piqa/completions/ben_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,039 copying build/lib/lm_eval/tasks/global_piqa/completions/cmn_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,041 copying build/lib/lm_eval/tasks/global_piqa/completions/ell_grek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,043 copying build/lib/lm_eval/tasks/global_piqa/completions/ekp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,045 copying build/lib/lm_eval/tasks/global_piqa/completions/srp_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,046 copying build/lib/lm_eval/tasks/global_piqa/completions/tur_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,048 copying build/lib/lm_eval/tasks/global_piqa/completions/uig_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,050 copying build/lib/lm_eval/tasks/global_piqa/completions/tgl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,052 copying build/lib/lm_eval/tasks/global_piqa/completions/arq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,054 copying build/lib/lm_eval/tasks/global_piqa/completions/cmn_hans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,056 copying build/lib/lm_eval/tasks/global_piqa/completions/swh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,058 copying build/lib/lm_eval/tasks/global_piqa/completions/arb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,060 copying build/lib/lm_eval/tasks/global_piqa/completions/mal_mlym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,061 copying build/lib/lm_eval/tasks/global_piqa/completions/rus_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,063 copying build/lib/lm_eval/tasks/global_piqa/completions/arz_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,065 copying build/lib/lm_eval/tasks/global_piqa/completions/por_latn_port.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,066 copying build/lib/lm_eval/tasks/global_piqa/completions/aeb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,068 copying build/lib/lm_eval/tasks/global_piqa/completions/nob_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,070 copying build/lib/lm_eval/tasks/global_piqa/completions/snd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,072 copying build/lib/lm_eval/tasks/global_piqa/completions/kan_knda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,073 copying build/lib/lm_eval/tasks/global_piqa/completions/nno_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,075 copying build/lib/lm_eval/tasks/global_piqa/completions/ben_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,077 copying build/lib/lm_eval/tasks/global_piqa/completions/ron_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,079 copying build/lib/lm_eval/tasks/global_piqa/completions/amh_ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,081 copying build/lib/lm_eval/tasks/global_piqa/completions/iso_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,083 copying build/lib/lm_eval/tasks/global_piqa/completions/als_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,085 copying build/lib/lm_eval/tasks/global_piqa/completions/ckb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,086 copying build/lib/lm_eval/tasks/global_piqa/completions/kir_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,088 copying build/lib/lm_eval/tasks/global_piqa/completions/kaz_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,090 copying build/lib/lm_eval/tasks/global_piqa/completions/npi_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,092 copying build/lib/lm_eval/tasks/global_piqa/completions/fao_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,094 copying build/lib/lm_eval/tasks/global_piqa/completions/luo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,096 copying build/lib/lm_eval/tasks/global_piqa/completions/bsk_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,098 copying build/lib/lm_eval/tasks/global_piqa/completions/azj_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,099 copying build/lib/lm_eval/tasks/global_piqa/completions/acm_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,101 copying build/lib/lm_eval/tasks/global_piqa/completions/hau_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,103 copying build/lib/lm_eval/tasks/global_piqa/completions/idu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,104 copying build/lib/lm_eval/tasks/global_piqa/completions/lin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,106 copying build/lib/lm_eval/tasks/global_piqa/completions/afb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,108 copying build/lib/lm_eval/tasks/global_piqa/completions/haw_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,110 copying build/lib/lm_eval/tasks/global_piqa/completions/ces_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,111 copying build/lib/lm_eval/tasks/global_piqa/completions/kor_hang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,113 copying build/lib/lm_eval/tasks/global_piqa/completions/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,115 copying build/lib/lm_eval/tasks/global_piqa/completions/slk_latn_sari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,117 copying build/lib/lm_eval/tasks/global_piqa/completions/fra_latn_cana.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,119 copying build/lib/lm_eval/tasks/global_piqa/completions/kat_geor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,120 copying build/lib/lm_eval/tasks/global_piqa/completions/mar_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,122 copying build/lib/lm_eval/tasks/global_piqa/completions/kin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,124 copying build/lib/lm_eval/tasks/global_piqa/completions/urd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,126 copying build/lib/lm_eval/tasks/global_piqa/completions/nld_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,128 copying build/lib/lm_eval/tasks/global_piqa/completions/deu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,130 copying build/lib/lm_eval/tasks/global_piqa/completions/pcm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,132 copying build/lib/lm_eval/tasks/global_piqa/completions/eng_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,133 copying build/lib/lm_eval/tasks/global_piqa/completions/urd_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,135 copying build/lib/lm_eval/tasks/global_piqa/completions/ary_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,137 copying build/lib/lm_eval/tasks/global_piqa/completions/mkd_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,139 copying build/lib/lm_eval/tasks/global_piqa/completions/pan_guru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,141 copying build/lib/lm_eval/tasks/global_piqa/completions/heb_hebr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,144 copying build/lib/lm_eval/tasks/global_piqa/completions/sin_sinh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,147 copying build/lib/lm_eval/tasks/global_piqa/completions/ars_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,149 copying build/lib/lm_eval/tasks/global_piqa/completions/spa_latn_spai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,152 copying build/lib/lm_eval/tasks/global_piqa/completions/asm_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,154 copying build/lib/lm_eval/tasks/global_piqa/completions/ibo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,156 copying build/lib/lm_eval/tasks/global_piqa/completions/bul_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,158 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_jord.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,159 copying build/lib/lm_eval/tasks/global_piqa/completions/hrv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,161 copying build/lib/lm_eval/tasks/global_piqa/completions/zsm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,163 copying build/lib/lm_eval/tasks/global_piqa/completions/pes_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,164 copying build/lib/lm_eval/tasks/global_piqa/completions/apc_arab_syri.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/completions 2026-05-11T15:53:03,169 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,170 copying build/lib/lm_eval/tasks/global_piqa/prompted/nag_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,171 copying build/lib/lm_eval/tasks/global_piqa/prompted/por_latn_braz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,173 copying build/lib/lm_eval/tasks/global_piqa/prompted/zul_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,175 copying build/lib/lm_eval/tasks/global_piqa/prompted/hin_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,177 copying build/lib/lm_eval/tasks/global_piqa/prompted/hye_armn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,178 copying build/lib/lm_eval/tasks/global_piqa/prompted/vie_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,180 copying build/lib/lm_eval/tasks/global_piqa/prompted/bho_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,182 copying build/lib/lm_eval/tasks/global_piqa/prompted/fin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,184 copying build/lib/lm_eval/tasks/global_piqa/prompted/bam_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,185 copying build/lib/lm_eval/tasks/global_piqa/prompted/urh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,187 copying build/lib/lm_eval/tasks/global_piqa/prompted/rwr_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,189 copying build/lib/lm_eval/tasks/global_piqa/prompted/spa_latn_mexi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,191 copying build/lib/lm_eval/tasks/global_piqa/prompted/acq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,193 copying build/lib/lm_eval/tasks/global_piqa/prompted/ita_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,194 copying build/lib/lm_eval/tasks/global_piqa/prompted/ekk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,196 copying build/lib/lm_eval/tasks/global_piqa/prompted/cat_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,198 copying build/lib/lm_eval/tasks/global_piqa/prompted/isl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,200 copying build/lib/lm_eval/tasks/global_piqa/prompted/_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,201 copying build/lib/lm_eval/tasks/global_piqa/prompted/ckm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,203 copying build/lib/lm_eval/tasks/global_piqa/prompted/yor_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,205 copying build/lib/lm_eval/tasks/global_piqa/prompted/uzn_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,206 copying build/lib/lm_eval/tasks/global_piqa/prompted/hun_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,208 copying build/lib/lm_eval/tasks/global_piqa/prompted/pol_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,210 copying build/lib/lm_eval/tasks/global_piqa/prompted/mni_mtei.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,212 copying build/lib/lm_eval/tasks/global_piqa/prompted/slk_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,214 copying build/lib/lm_eval/tasks/global_piqa/prompted/glg_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,215 copying build/lib/lm_eval/tasks/global_piqa/prompted/bel_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,217 copying build/lib/lm_eval/tasks/global_piqa/prompted/tel_telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,219 copying build/lib/lm_eval/tasks/global_piqa/prompted/swe_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,221 copying build/lib/lm_eval/tasks/global_piqa/prompted/yue_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,223 copying build/lib/lm_eval/tasks/global_piqa/prompted/jpn_jpan.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,224 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_pale.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,226 copying build/lib/lm_eval/tasks/global_piqa/prompted/snd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,228 copying build/lib/lm_eval/tasks/global_piqa/prompted/slv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,230 copying build/lib/lm_eval/tasks/global_piqa/prompted/dhd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,232 copying build/lib/lm_eval/tasks/global_piqa/prompted/bos_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,233 copying build/lib/lm_eval/tasks/global_piqa/prompted/slv_latn_cerk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,235 copying build/lib/lm_eval/tasks/global_piqa/prompted/ukr_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,237 copying build/lib/lm_eval/tasks/global_piqa/prompted/guj_gujr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,239 copying build/lib/lm_eval/tasks/global_piqa/prompted/tam_taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,241 copying build/lib/lm_eval/tasks/global_piqa/prompted/fra_latn_fran.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,242 copying build/lib/lm_eval/tasks/global_piqa/prompted/mni_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,244 copying build/lib/lm_eval/tasks/global_piqa/prompted/lit_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,246 copying build/lib/lm_eval/tasks/global_piqa/prompted/jav_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,248 copying build/lib/lm_eval/tasks/global_piqa/prompted/_global_piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,250 copying build/lib/lm_eval/tasks/global_piqa/prompted/tha_thai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,252 copying build/lib/lm_eval/tasks/global_piqa/prompted/srp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,254 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_leba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,256 copying build/lib/lm_eval/tasks/global_piqa/prompted/spa_latn_peru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,258 copying build/lib/lm_eval/tasks/global_piqa/prompted/ind_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,259 copying build/lib/lm_eval/tasks/global_piqa/prompted/ben_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,261 copying build/lib/lm_eval/tasks/global_piqa/prompted/cmn_hant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,263 copying build/lib/lm_eval/tasks/global_piqa/prompted/ell_grek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,265 copying build/lib/lm_eval/tasks/global_piqa/prompted/ekp_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,267 copying build/lib/lm_eval/tasks/global_piqa/prompted/srp_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,269 copying build/lib/lm_eval/tasks/global_piqa/prompted/tur_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,271 copying build/lib/lm_eval/tasks/global_piqa/prompted/uig_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,272 copying build/lib/lm_eval/tasks/global_piqa/prompted/tgl_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,274 copying build/lib/lm_eval/tasks/global_piqa/prompted/arq_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,276 copying build/lib/lm_eval/tasks/global_piqa/prompted/cmn_hans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,277 copying build/lib/lm_eval/tasks/global_piqa/prompted/swh_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,279 copying build/lib/lm_eval/tasks/global_piqa/prompted/arb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,281 copying build/lib/lm_eval/tasks/global_piqa/prompted/mal_mlym.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,283 copying build/lib/lm_eval/tasks/global_piqa/prompted/rus_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,285 copying build/lib/lm_eval/tasks/global_piqa/prompted/arz_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,287 copying build/lib/lm_eval/tasks/global_piqa/prompted/por_latn_port.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,288 copying build/lib/lm_eval/tasks/global_piqa/prompted/aeb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,290 copying build/lib/lm_eval/tasks/global_piqa/prompted/nob_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,292 copying build/lib/lm_eval/tasks/global_piqa/prompted/snd_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,293 copying build/lib/lm_eval/tasks/global_piqa/prompted/kan_knda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,295 copying build/lib/lm_eval/tasks/global_piqa/prompted/nno_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,297 copying build/lib/lm_eval/tasks/global_piqa/prompted/ben_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,309 copying build/lib/lm_eval/tasks/global_piqa/prompted/ron_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,310 copying build/lib/lm_eval/tasks/global_piqa/prompted/amh_ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,312 copying build/lib/lm_eval/tasks/global_piqa/prompted/iso_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,314 copying build/lib/lm_eval/tasks/global_piqa/prompted/als_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,316 copying build/lib/lm_eval/tasks/global_piqa/prompted/ckb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,317 copying build/lib/lm_eval/tasks/global_piqa/prompted/kir_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,319 copying build/lib/lm_eval/tasks/global_piqa/prompted/kaz_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,321 copying build/lib/lm_eval/tasks/global_piqa/prompted/npi_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,323 copying build/lib/lm_eval/tasks/global_piqa/prompted/fao_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,325 copying build/lib/lm_eval/tasks/global_piqa/prompted/luo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,326 copying build/lib/lm_eval/tasks/global_piqa/prompted/bsk_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,328 copying build/lib/lm_eval/tasks/global_piqa/prompted/azj_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,330 copying build/lib/lm_eval/tasks/global_piqa/prompted/acm_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,332 copying build/lib/lm_eval/tasks/global_piqa/prompted/hau_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,334 copying build/lib/lm_eval/tasks/global_piqa/prompted/idu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,336 copying build/lib/lm_eval/tasks/global_piqa/prompted/lin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,338 copying build/lib/lm_eval/tasks/global_piqa/prompted/afb_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,339 copying build/lib/lm_eval/tasks/global_piqa/prompted/haw_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,341 copying build/lib/lm_eval/tasks/global_piqa/prompted/ces_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,343 copying build/lib/lm_eval/tasks/global_piqa/prompted/kor_hang.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,344 copying build/lib/lm_eval/tasks/global_piqa/prompted/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,346 copying build/lib/lm_eval/tasks/global_piqa/prompted/slk_latn_sari.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,348 copying build/lib/lm_eval/tasks/global_piqa/prompted/fra_latn_cana.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,350 copying build/lib/lm_eval/tasks/global_piqa/prompted/kat_geor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,352 copying build/lib/lm_eval/tasks/global_piqa/prompted/mar_deva.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,354 copying build/lib/lm_eval/tasks/global_piqa/prompted/kin_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,355 copying build/lib/lm_eval/tasks/global_piqa/prompted/urd_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,357 copying build/lib/lm_eval/tasks/global_piqa/prompted/nld_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,359 copying build/lib/lm_eval/tasks/global_piqa/prompted/deu_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,361 copying build/lib/lm_eval/tasks/global_piqa/prompted/pcm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,362 copying build/lib/lm_eval/tasks/global_piqa/prompted/eng_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,364 copying build/lib/lm_eval/tasks/global_piqa/prompted/urd_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,366 copying build/lib/lm_eval/tasks/global_piqa/prompted/ary_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,368 copying build/lib/lm_eval/tasks/global_piqa/prompted/mkd_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,369 copying build/lib/lm_eval/tasks/global_piqa/prompted/pan_guru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,371 copying build/lib/lm_eval/tasks/global_piqa/prompted/heb_hebr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,373 copying build/lib/lm_eval/tasks/global_piqa/prompted/sin_sinh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,375 copying build/lib/lm_eval/tasks/global_piqa/prompted/ars_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,377 copying build/lib/lm_eval/tasks/global_piqa/prompted/spa_latn_spai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,379 copying build/lib/lm_eval/tasks/global_piqa/prompted/asm_beng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,381 copying build/lib/lm_eval/tasks/global_piqa/prompted/ibo_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,383 copying build/lib/lm_eval/tasks/global_piqa/prompted/bul_cyrl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,385 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_jord.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,386 copying build/lib/lm_eval/tasks/global_piqa/prompted/hrv_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,388 copying build/lib/lm_eval/tasks/global_piqa/prompted/zsm_latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,390 copying build/lib/lm_eval/tasks/global_piqa/prompted/pes_arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,392 copying build/lib/lm_eval/tasks/global_piqa/prompted/apc_arab_syri.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa/prompted 2026-05-11T15:53:03,393 copying build/lib/lm_eval/tasks/global_piqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_piqa 2026-05-11T15:53:03,396 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bhs 2026-05-11T15:53:03,397 copying build/lib/lm_eval/tasks/bhs/basque-S-S_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,399 copying build/lib/lm_eval/tasks/bhs/hindi-S_PossPRN_PossN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,401 copying build/lib/lm_eval/tasks/bhs/bhs_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,403 copying build/lib/lm_eval/tasks/bhs/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,405 copying build/lib/lm_eval/tasks/bhs/bhs_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,406 copying build/lib/lm_eval/tasks/bhs/basque-S-S_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,408 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_ni_AN.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,410 copying build/lib/lm_eval/tasks/bhs/hindi-S_ne_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,412 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,413 copying build/lib/lm_eval/tasks/bhs/basque-S-IO_S_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,415 copying build/lib/lm_eval/tasks/bhs/basque-IO-IO_S_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,417 copying build/lib/lm_eval/tasks/bhs/hindi-S_ne_PossPRN_PossN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,419 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,420 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,422 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_V_ni_AN.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,424 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_ni_A.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,425 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_ni_A.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,427 copying build/lib/lm_eval/tasks/bhs/basque-DO-S_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,429 copying build/lib/lm_eval/tasks/bhs/basque-DO-S_IO_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,431 copying build/lib/lm_eval/tasks/bhs/basque-IO-S_IO_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,433 copying build/lib/lm_eval/tasks/bhs/bhs_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,435 copying build/lib/lm_eval/tasks/bhs/hindi-S_ne_PossPRN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,436 copying build/lib/lm_eval/tasks/bhs/hindi-S_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,438 copying build/lib/lm_eval/tasks/bhs/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,440 copying build/lib/lm_eval/tasks/bhs/basque-S-S_IO_DO_V_AUX.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,442 copying build/lib/lm_eval/tasks/bhs/hindi-S_PossPRN_O_V.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,444 copying build/lib/lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V1_V2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bhs 2026-05-11T15:53:03,446 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada_cloze 2026-05-11T15:53:03,447 copying build/lib/lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_cloze 2026-05-11T15:53:03,449 copying build/lib/lm_eval/tasks/lambada_cloze/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_cloze 2026-05-11T15:53:03,450 copying build/lib/lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada_cloze 2026-05-11T15:53:03,453 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_reading 2026-05-11T15:53:03,454 copying build/lib/lm_eval/tasks/eus_reading/eus_reading.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_reading 2026-05-11T15:53:03,455 copying build/lib/lm_eval/tasks/eus_reading/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_reading 2026-05-11T15:53:03,457 copying build/lib/lm_eval/tasks/eus_reading/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_reading 2026-05-11T15:53:03,460 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/longbench 2026-05-11T15:53:03,461 copying build/lib/lm_eval/tasks/longbench/trec_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,463 copying build/lib/lm_eval/tasks/longbench/multi_news.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,465 copying build/lib/lm_eval/tasks/longbench/_longbench_code.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,467 copying build/lib/lm_eval/tasks/longbench/repobench-p.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,469 copying build/lib/lm_eval/tasks/longbench/musique.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,471 copying build/lib/lm_eval/tasks/longbench/_longbench_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,473 copying build/lib/lm_eval/tasks/longbench/hotpotqa_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,475 copying build/lib/lm_eval/tasks/longbench/passage_count_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,477 copying build/lib/lm_eval/tasks/longbench/narrativeqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,479 copying build/lib/lm_eval/tasks/longbench/lcc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,481 copying build/lib/lm_eval/tasks/longbench/samsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,484 copying build/lib/lm_eval/tasks/longbench/gov_report.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,486 copying build/lib/lm_eval/tasks/longbench/multi_news_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,488 copying build/lib/lm_eval/tasks/longbench/_longbench_fewshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,490 copying build/lib/lm_eval/tasks/longbench/multifieldqa_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,492 copying build/lib/lm_eval/tasks/longbench/_longbench_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,495 copying build/lib/lm_eval/tasks/longbench/_longbench_synthetic_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,497 copying build/lib/lm_eval/tasks/longbench/_longbench_fewshot_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,499 copying build/lib/lm_eval/tasks/longbench/_longbench_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,501 copying build/lib/lm_eval/tasks/longbench/_longbench_multi_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,503 copying build/lib/lm_eval/tasks/longbench/triviaqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,505 copying build/lib/lm_eval/tasks/longbench/hotpotqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,507 copying build/lib/lm_eval/tasks/longbench/qmsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,509 copying build/lib/lm_eval/tasks/longbench/passage_count.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,511 copying build/lib/lm_eval/tasks/longbench/samsum_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,513 copying build/lib/lm_eval/tasks/longbench/vcsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,515 copying build/lib/lm_eval/tasks/longbench/_longbench_single_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,517 copying build/lib/lm_eval/tasks/longbench/qasper_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,519 copying build/lib/lm_eval/tasks/longbench/passage_retrieval_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,522 copying build/lib/lm_eval/tasks/longbench/passage_retrieval_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,524 copying build/lib/lm_eval/tasks/longbench/2wikimqa_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,526 copying build/lib/lm_eval/tasks/longbench/triviaqa_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,528 copying build/lib/lm_eval/tasks/longbench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,530 copying build/lib/lm_eval/tasks/longbench/_longbench_summarization_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,532 copying build/lib/lm_eval/tasks/longbench/dureader.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,535 copying build/lib/lm_eval/tasks/longbench/_longbench_code_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,537 copying build/lib/lm_eval/tasks/longbench/qasper.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,539 copying build/lib/lm_eval/tasks/longbench/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,542 copying build/lib/lm_eval/tasks/longbench/_longbench_synthetic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,544 copying build/lib/lm_eval/tasks/longbench/trec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,546 copying build/lib/lm_eval/tasks/longbench/repobench-p_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,548 copying build/lib/lm_eval/tasks/longbench/_longbench_summarization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,550 copying build/lib/lm_eval/tasks/longbench/passage_retrieval_en_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,552 copying build/lib/lm_eval/tasks/longbench/gov_report_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,554 copying build/lib/lm_eval/tasks/longbench/2wikimqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,556 copying build/lib/lm_eval/tasks/longbench/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,559 copying build/lib/lm_eval/tasks/longbench/lcc_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,561 copying build/lib/lm_eval/tasks/longbench/_longbench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,563 copying build/lib/lm_eval/tasks/longbench/multifieldqa_en_e.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,565 copying build/lib/lm_eval/tasks/longbench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,567 copying build/lib/lm_eval/tasks/longbench/lsht.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,569 copying build/lib/lm_eval/tasks/longbench/multifieldqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench 2026-05-11T15:53:03,572 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu 2026-05-11T15:53:03,574 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,576 copying build/lib/lm_eval/tasks/mmlu/continuation/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,579 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,581 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,583 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,585 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,588 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,590 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,592 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,594 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,596 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,598 copying build/lib/lm_eval/tasks/mmlu/continuation/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,600 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,602 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,604 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,606 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,608 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,610 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,612 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,614 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,617 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,619 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,621 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,623 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,625 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,627 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,629 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,631 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,633 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,635 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,638 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,640 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,642 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,644 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,646 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,649 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,651 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,653 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,655 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,657 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,659 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,661 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,663 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,665 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,667 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,669 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,671 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,673 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,675 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,677 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,679 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,682 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,684 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,686 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,688 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,690 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,692 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,694 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,696 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,699 copying build/lib/lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/continuation 2026-05-11T15:53:03,702 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,703 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,706 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,708 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,710 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,712 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,715 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,717 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,720 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,722 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,724 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,727 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,729 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,731 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,734 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,736 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,739 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,741 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,743 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,745 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,747 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,750 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,752 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,755 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,757 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,760 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,762 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,764 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,767 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,769 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,771 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,773 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,775 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,777 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,780 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,782 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,784 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,786 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,792 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,794 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,797 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,800 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,803 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,805 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,807 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,810 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,812 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,814 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,817 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,820 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,822 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,824 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,826 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,828 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,830 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,833 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,834 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,837 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,839 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,841 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,843 copying build/lib/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_fewshot 2026-05-11T15:53:03,846 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,847 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,849 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,851 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,853 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,854 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,856 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,858 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,860 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,862 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,864 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,865 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,867 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,868 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,870 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,872 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,873 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,875 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,877 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,879 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,880 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,882 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,884 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,885 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,887 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,889 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,890 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,892 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,894 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,896 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,897 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,899 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,901 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,903 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,905 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,907 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,908 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,910 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,912 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,913 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,915 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,917 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,918 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,920 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,922 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,923 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,925 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,927 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,929 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,930 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,932 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,934 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,935 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,937 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,939 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,941 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,943 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,944 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,946 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,947 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,949 copying build/lib/lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_cot_zeroshot 2026-05-11T15:53:03,951 copying build/lib/lm_eval/tasks/mmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu 2026-05-11T15:53:03,954 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,955 copying build/lib/lm_eval/tasks/mmlu/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,957 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,959 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,960 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,962 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,964 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,965 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,967 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,969 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,970 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,972 copying build/lib/lm_eval/tasks/mmlu/generative/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,974 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,976 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,977 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,979 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,981 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,983 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,985 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,987 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,988 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,990 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,991 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,993 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,995 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,997 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:03,999 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,000 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,002 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,004 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,005 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,007 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,009 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,011 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,013 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,014 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,016 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,018 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,019 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,021 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,023 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,025 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,027 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,028 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,030 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,032 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,033 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,035 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,037 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,038 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,040 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,042 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,043 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,045 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,047 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,049 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,051 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,052 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,054 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,056 copying build/lib/lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/generative 2026-05-11T15:53:04,058 copying build/lib/lm_eval/tasks/mmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu 2026-05-11T15:53:04,061 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,062 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,064 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,066 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,068 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,069 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,071 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,073 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,074 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,076 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,078 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,080 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,081 copying build/lib/lm_eval/tasks/mmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,083 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,085 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,087 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,088 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,090 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,092 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,094 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,095 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,097 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,099 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,101 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,102 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,104 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,106 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,108 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,110 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,112 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,113 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,115 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,117 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,119 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,120 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,122 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,124 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,126 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,127 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,129 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,131 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,132 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,134 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,136 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,138 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,140 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,142 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,143 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,145 copying build/lib/lm_eval/tasks/mmlu/default/_mmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,147 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,149 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,151 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,153 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,154 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,156 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,158 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,160 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,162 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,164 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,165 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,167 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,169 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,171 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,173 copying build/lib/lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/default 2026-05-11T15:53:04,175 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot 2026-05-11T15:53:04,178 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,179 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,181 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,183 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,185 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,187 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,189 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,191 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,193 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,195 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,197 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,199 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,201 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,202 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,204 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,206 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,208 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,210 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,212 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,213 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,215 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,217 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,219 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,221 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,222 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,224 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,226 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,228 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,229 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,231 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,233 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,235 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,236 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,238 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,240 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,242 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,244 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,246 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,248 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,250 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,252 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,253 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,255 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,256 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,258 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,260 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,261 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,263 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,265 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,266 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,268 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,270 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,272 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,273 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,275 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,277 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,279 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,280 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,282 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,284 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,285 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/generative 2026-05-11T15:53:04,289 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,290 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,292 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,293 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,295 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,297 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,299 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,300 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,302 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,304 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,305 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,307 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,309 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,311 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,312 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,314 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,316 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,318 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,320 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,322 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,323 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,325 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,326 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,328 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,330 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,331 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,333 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,335 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,337 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,338 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,340 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,342 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,343 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,345 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,347 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,349 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,350 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,352 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,354 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,356 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,357 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,359 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,361 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,362 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,364 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,366 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,367 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,369 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,371 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,372 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,374 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,376 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,377 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,379 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,381 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,383 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,385 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,386 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,388 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,390 copying build/lib/lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu/flan_n_shot/loglikelihood 2026-05-11T15:53:04,392 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/headqa 2026-05-11T15:53:04,393 copying build/lib/lm_eval/tasks/headqa/headqa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/headqa 2026-05-11T15:53:04,395 copying build/lib/lm_eval/tasks/headqa/headqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/headqa 2026-05-11T15:53:04,397 copying build/lib/lm_eval/tasks/headqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/headqa 2026-05-11T15:53:04,400 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp 2026-05-11T15:53:04,401 copying build/lib/lm_eval/tasks/blimp/intransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,403 copying build/lib/lm_eval/tasks/blimp/generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,404 copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,406 copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,408 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,409 copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,411 copying build/lib/lm_eval/tasks/blimp/anaphor_number_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,412 copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,414 copying build/lib/lm_eval/tasks/blimp/left_branch_island_echo_question.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,416 copying build/lib/lm_eval/tasks/blimp/only_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,418 copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,419 copying build/lib/lm_eval/tasks/blimp/superlative_quantifiers_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,421 copying build/lib/lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,423 copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,424 copying build/lib/lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,426 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,428 copying build/lib/lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,430 copying build/lib/lm_eval/tasks/blimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,431 copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,433 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,435 copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,437 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,439 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,441 copying build/lib/lm_eval/tasks/blimp/existential_there_object_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,443 copying build/lib/lm_eval/tasks/blimp/left_branch_island_simple_question.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,444 copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,446 copying build/lib/lm_eval/tasks/blimp/anaphor_gender_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,448 copying build/lib/lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,450 copying build/lib/lm_eval/tasks/blimp/drop_argument.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,452 copying build/lib/lm_eval/tasks/blimp/animate_subject_trans.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,453 copying build/lib/lm_eval/tasks/blimp/sentential_subject_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,455 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,457 copying build/lib/lm_eval/tasks/blimp/principle_A_case_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,459 copying build/lib/lm_eval/tasks/blimp/inchoative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,461 copying build/lib/lm_eval/tasks/blimp/causative.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,462 copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,464 copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,466 copying build/lib/lm_eval/tasks/blimp/animate_subject_passive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,468 copying build/lib/lm_eval/tasks/blimp/tough_vs_raising_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,469 copying build/lib/lm_eval/tasks/blimp/passive_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,471 copying build/lib/lm_eval/tasks/blimp/adjunct_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,473 copying build/lib/lm_eval/tasks/blimp/_blimp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,475 copying build/lib/lm_eval/tasks/blimp/principle_A_case_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,476 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,478 copying build/lib/lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,480 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,482 copying build/lib/lm_eval/tasks/blimp/wh_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,484 copying build/lib/lm_eval/tasks/blimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,485 copying build/lib/lm_eval/tasks/blimp/expletive_it_object_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,487 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,489 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,491 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,492 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,494 copying build/lib/lm_eval/tasks/blimp/principle_A_domain_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,496 copying build/lib/lm_eval/tasks/blimp/complex_NP_island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,497 copying build/lib/lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,499 copying build/lib/lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,501 copying build/lib/lm_eval/tasks/blimp/principle_A_reconstruction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,503 copying build/lib/lm_eval/tasks/blimp/npi_present_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,504 copying build/lib/lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,506 copying build/lib/lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,508 copying build/lib/lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,509 copying build/lib/lm_eval/tasks/blimp/existential_there_subject_raising.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,511 copying build/lib/lm_eval/tasks/blimp/passive_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,513 copying build/lib/lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,515 copying build/lib/lm_eval/tasks/blimp/transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,517 copying build/lib/lm_eval/tasks/blimp/wh_questions_object_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,518 copying build/lib/lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,520 copying build/lib/lm_eval/tasks/blimp/only_npi_scope.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,522 copying build/lib/lm_eval/tasks/blimp/npi_present_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,524 copying build/lib/lm_eval/tasks/blimp/principle_A_c_command.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp 2026-05-11T15:53:04,526 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mela 2026-05-11T15:53:04,527 copying build/lib/lm_eval/tasks/mela/mela_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,529 copying build/lib/lm_eval/tasks/mela/mela_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,531 copying build/lib/lm_eval/tasks/mela/mela_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,532 copying build/lib/lm_eval/tasks/mela/mela_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,534 copying build/lib/lm_eval/tasks/mela/mela_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,536 copying build/lib/lm_eval/tasks/mela/mela_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,538 copying build/lib/lm_eval/tasks/mela/mela_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,539 copying build/lib/lm_eval/tasks/mela/mela_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,541 copying build/lib/lm_eval/tasks/mela/_mela.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,543 copying build/lib/lm_eval/tasks/mela/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,545 copying build/lib/lm_eval/tasks/mela/mela_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,546 copying build/lib/lm_eval/tasks/mela/mela_is.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mela 2026-05-11T15:53:04,549 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmmu 2026-05-11T15:53:04,550 copying build/lib/lm_eval/tasks/mmmu/mmmu_mechanical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,552 copying build/lib/lm_eval/tasks/mmmu/mmmu_basic_medical_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,554 copying build/lib/lm_eval/tasks/mmmu/mmmu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,555 copying build/lib/lm_eval/tasks/mmmu/mmmu_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,557 copying build/lib/lm_eval/tasks/mmmu/mmmu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,559 copying build/lib/lm_eval/tasks/mmmu/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,561 copying build/lib/lm_eval/tasks/mmmu/_health_and_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,562 copying build/lib/lm_eval/tasks/mmmu/mmmu_public_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,564 copying build/lib/lm_eval/tasks/mmmu/mmmu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,566 copying build/lib/lm_eval/tasks/mmmu/_mmmu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,568 copying build/lib/lm_eval/tasks/mmmu/mmmu_manage.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,570 copying build/lib/lm_eval/tasks/mmmu/mmmu_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,571 copying build/lib/lm_eval/tasks/mmmu/_art_and_design.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,573 copying build/lib/lm_eval/tasks/mmmu/mmmu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,575 copying build/lib/lm_eval/tasks/mmmu/mmmu_music.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,577 copying build/lib/lm_eval/tasks/mmmu/mmmu_finance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,579 copying build/lib/lm_eval/tasks/mmmu/mmmu_agriculture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,581 copying build/lib/lm_eval/tasks/mmmu/mmmu_materials.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,582 copying build/lib/lm_eval/tasks/mmmu/mmmu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,584 copying build/lib/lm_eval/tasks/mmmu/_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,586 copying build/lib/lm_eval/tasks/mmmu/mmmu_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,588 copying build/lib/lm_eval/tasks/mmmu/mmmu_energy_and_power.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,590 copying build/lib/lm_eval/tasks/mmmu/mmmu_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,592 copying build/lib/lm_eval/tasks/mmmu/mmmu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,594 copying build/lib/lm_eval/tasks/mmmu/mmmu_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,596 copying build/lib/lm_eval/tasks/mmmu/_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,597 copying build/lib/lm_eval/tasks/mmmu/mmmu_art.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,599 copying build/lib/lm_eval/tasks/mmmu/_tech_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,601 copying build/lib/lm_eval/tasks/mmmu/mmmu_diagnostics_and_laboratory_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,603 copying build/lib/lm_eval/tasks/mmmu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,605 copying build/lib/lm_eval/tasks/mmmu/mmmu_electronics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,607 copying build/lib/lm_eval/tasks/mmmu/_humanities_and_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,609 copying build/lib/lm_eval/tasks/mmmu/mmmu_pharmacy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,610 copying build/lib/lm_eval/tasks/mmmu/mmmu_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,612 copying build/lib/lm_eval/tasks/mmmu/mmmu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,614 copying build/lib/lm_eval/tasks/mmmu/mmmu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,616 copying build/lib/lm_eval/tasks/mmmu/mmmu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,617 copying build/lib/lm_eval/tasks/mmmu/mmmu_art_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,619 copying build/lib/lm_eval/tasks/mmmu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,622 copying build/lib/lm_eval/tasks/mmmu/mmmu_architecture_and_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmmu 2026-05-11T15:53:04,624 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/histoires_morales 2026-05-11T15:53:04,625 copying build/lib/lm_eval/tasks/histoires_morales/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/histoires_morales 2026-05-11T15:53:04,627 copying build/lib/lm_eval/tasks/histoires_morales/histoires_morales.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/histoires_morales 2026-05-11T15:53:04,629 copying build/lib/lm_eval/tasks/histoires_morales/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/histoires_morales 2026-05-11T15:53:04,632 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/prost 2026-05-11T15:53:04,633 copying build/lib/lm_eval/tasks/prost/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/prost 2026-05-11T15:53:04,635 copying build/lib/lm_eval/tasks/prost/corypaik_prost.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/prost 2026-05-11T15:53:04,637 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mastermind 2026-05-11T15:53:04,638 copying build/lib/lm_eval/tasks/mastermind/mastermind_46_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,640 copying build/lib/lm_eval/tasks/mastermind/mastermind_46_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,642 copying build/lib/lm_eval/tasks/mastermind/mastermind_35_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,644 copying build/lib/lm_eval/tasks/mastermind/mastermind_35_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,646 copying build/lib/lm_eval/tasks/mastermind/mastermind_24_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,648 copying build/lib/lm_eval/tasks/mastermind/mastermind_24_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,650 copying build/lib/lm_eval/tasks/mastermind/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mastermind 2026-05-11T15:53:04,652 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/anli 2026-05-11T15:53:04,653 copying build/lib/lm_eval/tasks/anli/anli_r1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T15:53:04,655 copying build/lib/lm_eval/tasks/anli/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T15:53:04,657 copying build/lib/lm_eval/tasks/anli/anli_r2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T15:53:04,659 copying build/lib/lm_eval/tasks/anli/anli_r3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/anli 2026-05-11T15:53:04,662 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/longbench2 2026-05-11T15:53:04,663 copying build/lib/lm_eval/tasks/longbench2/_longbench2_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,665 copying build/lib/lm_eval/tasks/longbench2/legal_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,667 copying build/lib/lm_eval/tasks/longbench2/lit_single_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,669 copying build/lib/lm_eval/tasks/longbench2/many_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,670 copying build/lib/lm_eval/tasks/longbench2/agent_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,673 copying build/lib/lm_eval/tasks/longbench2/graph.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,675 copying build/lib/lm_eval/tasks/longbench2/_longbench2_structured.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,677 copying build/lib/lm_eval/tasks/longbench2/table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,679 copying build/lib/lm_eval/tasks/longbench2/_longbench_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,680 copying build/lib/lm_eval/tasks/longbench2/legal_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,682 copying build/lib/lm_eval/tasks/longbench2/_longbench2_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,684 copying build/lib/lm_eval/tasks/longbench2/detective.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,686 copying build/lib/lm_eval/tasks/longbench2/longbench2_code.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,688 copying build/lib/lm_eval/tasks/longbench2/academic_single.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,690 copying build/lib/lm_eval/tasks/longbench2/user_guide.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,692 copying build/lib/lm_eval/tasks/longbench2/_longbench2_incontext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,694 copying build/lib/lm_eval/tasks/longbench2/_longbench2_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,695 copying build/lib/lm_eval/tasks/longbench2/news_multi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,697 copying build/lib/lm_eval/tasks/longbench2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,700 copying build/lib/lm_eval/tasks/longbench2/govt_multi_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,702 copying build/lib/lm_eval/tasks/longbench2/_longbench2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,704 copying build/lib/lm_eval/tasks/longbench2/dialogue_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,706 copying build/lib/lm_eval/tasks/longbench2/govt_single_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,707 copying build/lib/lm_eval/tasks/longbench2/academic_multi_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,709 copying build/lib/lm_eval/tasks/longbench2/translate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,711 copying build/lib/lm_eval/tasks/longbench2/fin_single_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,713 copying build/lib/lm_eval/tasks/longbench2/fin_multi_doc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,715 copying build/lib/lm_eval/tasks/longbench2/event_order.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/longbench2 2026-05-11T15:53:04,717 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,718 copying build/lib/lm_eval/tasks/kormedmcqa/pharm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,720 copying build/lib/lm_eval/tasks/kormedmcqa/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,722 copying build/lib/lm_eval/tasks/kormedmcqa/doctor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,724 copying build/lib/lm_eval/tasks/kormedmcqa/nurse.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,726 copying build/lib/lm_eval/tasks/kormedmcqa/_kormedmcqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,727 copying build/lib/lm_eval/tasks/kormedmcqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,730 copying build/lib/lm_eval/tasks/kormedmcqa/dentist.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kormedmcqa 2026-05-11T15:53:04,732 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu 2026-05-11T15:53:04,733 copying build/lib/lm_eval/tasks/afrimmlu/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T15:53:04,736 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate 2026-05-11T15:53:04,738 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,739 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,741 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,743 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,745 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,746 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,749 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,750 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,752 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,754 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,756 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,757 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,759 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,761 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,763 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,765 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,766 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,768 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,770 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,772 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_2 2026-05-11T15:53:04,774 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,776 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,778 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,780 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,781 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,783 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,785 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,787 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,789 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,791 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,793 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,795 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,797 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,799 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,801 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,802 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,804 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,806 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,808 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,810 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_1 2026-05-11T15:53:04,812 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,813 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,815 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,817 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,819 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,821 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,823 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,825 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,826 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,829 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,831 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,832 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,834 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,836 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,838 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,840 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,842 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,844 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,846 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,847 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_5 2026-05-11T15:53:04,849 copying build/lib/lm_eval/tasks/afrimmlu/translate/afrimmlu_tt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate 2026-05-11T15:53:04,852 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,853 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,855 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,856 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,858 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,860 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,862 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,864 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,866 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,867 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,869 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,871 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,873 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,875 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,877 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,879 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,881 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,883 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,884 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,886 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_3 2026-05-11T15:53:04,888 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,889 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,891 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,893 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,895 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,897 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,899 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,901 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,902 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,904 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,906 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,908 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,909 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,911 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,913 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,915 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,917 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,919 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,920 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,922 copying build/lib/lm_eval/tasks/afrimmlu/translate/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/translate/prompt_4 2026-05-11T15:53:04,924 copying build/lib/lm_eval/tasks/afrimmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T15:53:04,926 copying build/lib/lm_eval/tasks/afrimmlu/fewshot.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T15:53:04,928 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct 2026-05-11T15:53:04,930 copying build/lib/lm_eval/tasks/afrimmlu/direct/afrimmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct 2026-05-11T15:53:04,932 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,933 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,935 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,937 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,939 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,941 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,943 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,945 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,947 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,949 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,951 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,953 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,954 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,956 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,958 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,960 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,962 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,964 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,966 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,967 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,969 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_2 2026-05-11T15:53:04,971 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,972 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,974 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,976 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,978 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,980 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,982 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,984 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,986 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,988 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,989 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,991 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,993 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,995 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,997 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:04,999 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:05,001 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:05,003 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:05,005 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:05,007 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:05,009 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_1 2026-05-11T15:53:05,011 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,012 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,015 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,016 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,018 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,020 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,022 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,024 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,026 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,028 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,030 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,032 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,034 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,036 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,038 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,040 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,042 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,044 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,046 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,048 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,049 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_5 2026-05-11T15:53:05,052 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,053 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,055 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,056 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,058 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,060 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,061 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,063 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,065 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,067 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,069 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,071 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,073 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,075 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,077 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,079 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,081 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,082 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,084 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,086 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,088 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_3 2026-05-11T15:53:05,091 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,092 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,094 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,096 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,097 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,099 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,101 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,103 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,105 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,106 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,108 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,110 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,112 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,114 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,115 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,117 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,119 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,121 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,123 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,125 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,127 copying build/lib/lm_eval/tasks/afrimmlu/direct/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu/direct/prompt_4 2026-05-11T15:53:05,129 copying build/lib/lm_eval/tasks/afrimmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimmlu 2026-05-11T15:53:05,131 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mbpp 2026-05-11T15:53:05,133 copying build/lib/lm_eval/tasks/mbpp/mbpp_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T15:53:05,135 copying build/lib/lm_eval/tasks/mbpp/mbpp_plus_instruct.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T15:53:05,137 copying build/lib/lm_eval/tasks/mbpp/mbpp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T15:53:05,139 copying build/lib/lm_eval/tasks/mbpp/mbpp_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T15:53:05,141 copying build/lib/lm_eval/tasks/mbpp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T15:53:05,143 copying build/lib/lm_eval/tasks/mbpp/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mbpp 2026-05-11T15:53:05,147 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,148 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,150 copying build/lib/lm_eval/tasks/blimp_nl/determiners__geen_expletive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,152 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,153 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,155 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,157 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,159 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__argument_number_in_transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,161 copying build/lib/lm_eval/tasks/blimp_nl/complementive__position_verb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,162 copying build/lib/lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,164 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,166 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,168 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,170 copying build/lib/lm_eval/tasks/blimp_nl/complementive__position_adverb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,172 copying build/lib/lm_eval/tasks/blimp_nl/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,174 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,175 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__perfect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,177 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,179 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,181 copying build/lib/lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,183 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,184 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,186 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,188 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,190 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,191 copying build/lib/lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,193 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,195 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,197 copying build/lib/lm_eval/tasks/blimp_nl/adpositional_phrases__argument_r_extraction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,199 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__order_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,200 copying build/lib/lm_eval/tasks/blimp_nl/verb_second__order_main.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,202 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,204 copying build/lib/lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,206 copying build/lib/lm_eval/tasks/blimp_nl/determiners__geen_scrambling_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,208 copying build/lib/lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,210 copying build/lib/lm_eval/tasks/blimp_nl/adpositional_phrases__argument_scrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,211 copying build/lib/lm_eval/tasks/blimp_nl/anaphor_agreement__number.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,213 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__stranding_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,215 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,217 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,219 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,221 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,223 copying build/lib/lm_eval/tasks/blimp_nl/relativization__island.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,224 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,226 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__stranding_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,228 copying build/lib/lm_eval/tasks/blimp_nl/extraposition__adjectival_adverbial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,230 copying build/lib/lm_eval/tasks/blimp_nl/complementive__transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,232 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,234 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,235 copying build/lib/lm_eval/tasks/blimp_nl/anaphor_agreement__person.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,237 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,239 copying build/lib/lm_eval/tasks/blimp_nl/binding_principle_a__monomorphemic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,241 copying build/lib/lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,242 copying build/lib/lm_eval/tasks/blimp_nl/passive__aci.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,244 copying build/lib/lm_eval/tasks/blimp_nl/r_words__adverbial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,246 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__resumptive_prolepsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,248 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,249 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,251 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__order_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,253 copying build/lib/lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,255 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,257 copying build/lib/lm_eval/tasks/blimp_nl/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,259 copying build/lib/lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,261 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,263 copying build/lib/lm_eval/tasks/blimp_nl/complementive__ditransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,265 copying build/lib/lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,266 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__complementizer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,268 copying build/lib/lm_eval/tasks/blimp_nl/blimp_nl_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,271 copying build/lib/lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,272 copying build/lib/lm_eval/tasks/blimp_nl/extraposition__adjectival_supplementive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,274 copying build/lib/lm_eval/tasks/blimp_nl/adverbial_modification__position_type.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,276 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,278 copying build/lib/lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,280 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__superiority.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,281 copying build/lib/lm_eval/tasks/blimp_nl/extraposition__argument_nominal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,283 copying build/lib/lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,285 copying build/lib/lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,286 copying build/lib/lm_eval/tasks/blimp_nl/crossing_dependencies__cross_dependency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,288 copying build/lib/lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,290 copying build/lib/lm_eval/tasks/blimp_nl/binding_principle_a__c_command.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,291 copying build/lib/lm_eval/tasks/blimp_nl/determiners__geen_scrambling_1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,293 copying build/lib/lm_eval/tasks/blimp_nl/determiners__negative_polarity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,295 copying build/lib/lm_eval/tasks/blimp_nl/complementive__intransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,297 copying build/lib/lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,299 copying build/lib/lm_eval/tasks/blimp_nl/finite_argument_clause__perception_dat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,300 copying build/lib/lm_eval/tasks/blimp_nl/passive__impersonal.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,302 copying build/lib/lm_eval/tasks/blimp_nl/adverbial_modification__position_proform.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,304 copying build/lib/lm_eval/tasks/blimp_nl/argument_structure__argument_number_ditransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/blimp_nl 2026-05-11T15:53:05,307 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/moral_stories 2026-05-11T15:53:05,308 copying build/lib/lm_eval/tasks/moral_stories/moral_stories.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/moral_stories 2026-05-11T15:53:05,310 copying build/lib/lm_eval/tasks/moral_stories/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/moral_stories 2026-05-11T15:53:05,312 copying build/lib/lm_eval/tasks/moral_stories/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/moral_stories 2026-05-11T15:53:05,314 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lingoly 2026-05-11T15:53:05,315 copying build/lib/lm_eval/tasks/lingoly/lingoly_context.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T15:53:05,317 copying build/lib/lm_eval/tasks/lingoly/lingoly_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T15:53:05,319 copying build/lib/lm_eval/tasks/lingoly/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T15:53:05,321 copying build/lib/lm_eval/tasks/lingoly/lingoly_nocontext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T15:53:05,323 copying build/lib/lm_eval/tasks/lingoly/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T15:53:05,325 copying build/lib/lm_eval/tasks/lingoly/script.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lingoly 2026-05-11T15:53:05,328 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa2 2026-05-11T15:53:05,329 copying build/lib/lm_eval/tasks/logiqa2/logieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T15:53:05,331 copying build/lib/lm_eval/tasks/logiqa2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T15:53:05,333 copying build/lib/lm_eval/tasks/logiqa2/logiqa2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T15:53:05,334 copying build/lib/lm_eval/tasks/logiqa2/utils_logiqa2.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa2 2026-05-11T15:53:05,337 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/fld 2026-05-11T15:53:05,338 copying build/lib/lm_eval/tasks/fld/fld_star.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T15:53:05,340 copying build/lib/lm_eval/tasks/fld/fld_logical_formula_star.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T15:53:05,342 copying build/lib/lm_eval/tasks/fld/fld_logical_formula_default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T15:53:05,344 copying build/lib/lm_eval/tasks/fld/fld_default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T15:53:05,347 copying build/lib/lm_eval/tasks/fld/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/fld 2026-05-11T15:53:05,352 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench 2026-05-11T15:53:05,353 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,355 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/translation_dr_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,357 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/transliteration_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,360 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/transliteration_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,362 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,364 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/translation_ar_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,366 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/transliteration_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,368 copying build/lib/lm_eval/tasks/darija_bench/darija_transliteration/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_transliteration 2026-05-11T15:53:05,370 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,372 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msac.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,374 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_myc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,376 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,378 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_electrom.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,380 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msda.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,382 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/default_darija_sentiment_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,383 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_mac.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,385 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,387 copying build/lib/lm_eval/tasks/darija_bench/darija_sentiment/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_sentiment 2026-05-11T15:53:05,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,390 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,392 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,394 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,395 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_en_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,397 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,399 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,401 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,402 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,404 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,406 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,407 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,409 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,411 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_msa_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,413 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,414 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/translation_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,416 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_dr_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,418 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,420 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,421 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/seed_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,423 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_fr_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,424 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_fr_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,426 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,428 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,430 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_dr_msa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,431 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_en_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,433 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/translation_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,435 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_msa_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,437 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,439 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/madar_translation_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,440 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/flores_translation_msa_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,442 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,444 copying build/lib/lm_eval/tasks/darija_bench/darija_translation/doda_translation_en_dr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_translation 2026-05-11T15:53:05,446 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:53:05,447 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/summarization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:53:05,449 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:53:05,451 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/summarization_darija.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:53:05,453 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/summarization_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:53:05,455 copying build/lib/lm_eval/tasks/darija_bench/darija_summarization/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench/darija_summarization 2026-05-11T15:53:05,457 copying build/lib/lm_eval/tasks/darija_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/darija_bench 2026-05-11T15:53:05,460 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turkishmmlu 2026-05-11T15:53:05,461 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,462 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/History.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,465 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Turkish_Language_and_Literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,467 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,469 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,472 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/_turkishmmlu_cot_default_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,473 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,475 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,478 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,480 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,482 copying build/lib/lm_eval/tasks/turkishmmlu/config_cot/Religion_and_Ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config_cot 2026-05-11T15:53:05,485 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,486 copying build/lib/lm_eval/tasks/turkishmmlu/config/History.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,488 copying build/lib/lm_eval/tasks/turkishmmlu/config/Turkish_Language_and_Literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,490 copying build/lib/lm_eval/tasks/turkishmmlu/config/Geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,492 copying build/lib/lm_eval/tasks/turkishmmlu/config/_turkishmmlu_default_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,494 copying build/lib/lm_eval/tasks/turkishmmlu/config/Mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,495 copying build/lib/lm_eval/tasks/turkishmmlu/config/Biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,497 copying build/lib/lm_eval/tasks/turkishmmlu/config/Chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,499 copying build/lib/lm_eval/tasks/turkishmmlu/config/Philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,500 copying build/lib/lm_eval/tasks/turkishmmlu/config/Physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,502 copying build/lib/lm_eval/tasks/turkishmmlu/config/Religion_and_Ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu/config 2026-05-11T15:53:05,504 copying build/lib/lm_eval/tasks/turkishmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turkishmmlu 2026-05-11T15:53:05,507 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/polemo2 2026-05-11T15:53:05,508 copying build/lib/lm_eval/tasks/polemo2/polemo2_in.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/polemo2 2026-05-11T15:53:05,510 copying build/lib/lm_eval/tasks/polemo2/polemo2_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/polemo2 2026-05-11T15:53:05,512 copying build/lib/lm_eval/tasks/polemo2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/polemo2 2026-05-11T15:53:05,514 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,515 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,517 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,519 copying build/lib/lm_eval/tasks/mmlu_pro/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,521 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,522 copying build/lib/lm_eval/tasks/mmlu_pro/_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,524 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,526 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,527 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,529 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,531 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,533 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,534 copying build/lib/lm_eval/tasks/mmlu_pro/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,537 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,539 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,541 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,543 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,544 copying build/lib/lm_eval/tasks/mmlu_pro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,546 copying build/lib/lm_eval/tasks/mmlu_pro/mmlu_pro_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu_pro 2026-05-11T15:53:05,549 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard 2026-05-11T15:53:05,551 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,552 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/causal_judgement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,554 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/salient_translation_error_detection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,556 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/navigate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,558 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,560 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,562 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/temporal_sequences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,564 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_three_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,565 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/_leaderboard_bbh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,567 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/geometric_shapes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,569 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/boolean_expressions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,571 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,572 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/_fewshot_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,574 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/date_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,576 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/web_of_lies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,578 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/snarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,579 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/reasoning_about_colored_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,581 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,583 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/hyperbaton.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,585 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_five_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,587 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/sports_understanding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,589 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/object_counting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,591 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/disambiguation_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,593 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_seven_objects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,595 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/penguins_in_a_table.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,597 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/movie_recommendation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,599 copying build/lib/lm_eval/tasks/leaderboard/bbh_mc/ruin_names.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/bbh_mc 2026-05-11T15:53:05,601 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,602 copying build/lib/lm_eval/tasks/leaderboard/math/math_prealgebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,604 copying build/lib/lm_eval/tasks/leaderboard/math/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,606 copying build/lib/lm_eval/tasks/leaderboard/math/math_geometry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,608 copying build/lib/lm_eval/tasks/leaderboard/math/math_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,609 copying build/lib/lm_eval/tasks/leaderboard/math/math_intermediate_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,611 copying build/lib/lm_eval/tasks/leaderboard/math/math_counting_and_prob.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,613 copying build/lib/lm_eval/tasks/leaderboard/math/math_precalculus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,614 copying build/lib/lm_eval/tasks/leaderboard/math/_leaderboard_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,616 copying build/lib/lm_eval/tasks/leaderboard/math/math_num_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,618 copying build/lib/lm_eval/tasks/leaderboard/math/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/math 2026-05-11T15:53:05,620 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T15:53:05,621 copying build/lib/lm_eval/tasks/leaderboard/mmlu_pro/mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T15:53:05,623 copying build/lib/lm_eval/tasks/leaderboard/mmlu_pro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/mmlu_pro 2026-05-11T15:53:05,625 copying build/lib/lm_eval/tasks/leaderboard/leaderboard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard 2026-05-11T15:53:05,628 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,629 copying build/lib/lm_eval/tasks/leaderboard/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,631 copying build/lib/lm_eval/tasks/leaderboard/ifeval/_leaderboard_instruction_following.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,633 copying build/lib/lm_eval/tasks/leaderboard/ifeval/instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,635 copying build/lib/lm_eval/tasks/leaderboard/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,638 copying build/lib/lm_eval/tasks/leaderboard/ifeval/ifeval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,640 copying build/lib/lm_eval/tasks/leaderboard/ifeval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/ifeval 2026-05-11T15:53:05,642 copying build/lib/lm_eval/tasks/leaderboard/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard 2026-05-11T15:53:05,645 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,646 copying build/lib/lm_eval/tasks/leaderboard/musr/musr_murder_mysteries.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,648 copying build/lib/lm_eval/tasks/leaderboard/musr/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,650 copying build/lib/lm_eval/tasks/leaderboard/musr/_musr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,652 copying build/lib/lm_eval/tasks/leaderboard/musr/musr_team_allocation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,654 copying build/lib/lm_eval/tasks/leaderboard/musr/musr_object_placements.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,656 copying build/lib/lm_eval/tasks/leaderboard/musr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/musr 2026-05-11T15:53:05,658 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,659 copying build/lib/lm_eval/tasks/leaderboard/gpqa/_leaderboard_gpqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,661 copying build/lib/lm_eval/tasks/leaderboard/gpqa/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,663 copying build/lib/lm_eval/tasks/leaderboard/gpqa/gpqa_extended_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,664 copying build/lib/lm_eval/tasks/leaderboard/gpqa/gpqa_main_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,666 copying build/lib/lm_eval/tasks/leaderboard/gpqa/gpqa_diamond_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,668 copying build/lib/lm_eval/tasks/leaderboard/gpqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/leaderboard/gpqa 2026-05-11T15:53:05,670 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/sciq 2026-05-11T15:53:05,671 copying build/lib/lm_eval/tasks/sciq/sciq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/sciq 2026-05-11T15:53:05,673 copying build/lib/lm_eval/tasks/sciq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/sciq 2026-05-11T15:53:05,675 copying build/lib/lm_eval/tasks/_index.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T15:53:05,679 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ceval 2026-05-11T15:53:05,680 copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_network.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,682 copying build/lib/lm_eval/tasks/ceval/ceval-valid_physician.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,683 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,685 copying build/lib/lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,687 copying build/lib/lm_eval/tasks/ceval/ceval-valid_marxism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,689 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,691 copying build/lib/lm_eval/tasks/ceval/ceval-valid_accountant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,693 copying build/lib/lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,694 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,696 copying build/lib/lm_eval/tasks/ceval/ceval-valid_sports_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,698 copying build/lib/lm_eval/tasks/ceval/ceval-valid_operating_system.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,700 copying build/lib/lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,702 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,704 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,706 copying build/lib/lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,707 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,709 copying build/lib/lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,711 copying build/lib/lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,713 copying build/lib/lm_eval/tasks/ceval/ceval-valid_art_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,714 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,716 copying build/lib/lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,718 copying build/lib/lm_eval/tasks/ceval/ceval-valid_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,720 copying build/lib/lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,722 copying build/lib/lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,723 copying build/lib/lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,725 copying build/lib/lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,727 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,729 copying build/lib/lm_eval/tasks/ceval/_default_ceval_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,731 copying build/lib/lm_eval/tasks/ceval/ceval-valid_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,732 copying build/lib/lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,734 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,736 copying build/lib/lm_eval/tasks/ceval/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,738 copying build/lib/lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,740 copying build/lib/lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,742 copying build/lib/lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,744 copying build/lib/lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,745 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,747 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,749 copying build/lib/lm_eval/tasks/ceval/ceval-valid_education_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,751 copying build/lib/lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,753 copying build/lib/lm_eval/tasks/ceval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,755 copying build/lib/lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,757 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,759 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,760 copying build/lib/lm_eval/tasks/ceval/_ceval-valid.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,762 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_programming.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,764 copying build/lib/lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,766 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,767 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,769 copying build/lib/lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,771 copying build/lib/lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,772 copying build/lib/lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,774 copying build/lib/lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,776 copying build/lib/lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,778 copying build/lib/lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,780 copying build/lib/lm_eval/tasks/ceval/ceval-valid_business_administration.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ceval 2026-05-11T15:53:05,782 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aclue 2026-05-11T15:53:05,783 copying build/lib/lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,785 copying build/lib/lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,787 copying build/lib/lm_eval/tasks/aclue/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,789 copying build/lib/lm_eval/tasks/aclue/_aclue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,791 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,792 copying build/lib/lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,794 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,796 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,798 copying build/lib/lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,800 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,802 copying build/lib/lm_eval/tasks/aclue/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,804 copying build/lib/lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,806 copying build/lib/lm_eval/tasks/aclue/aclue_couplet_prediction.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,807 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,809 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,811 copying build/lib/lm_eval/tasks/aclue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,813 copying build/lib/lm_eval/tasks/aclue/aclue_reading_comprehension.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,814 copying build/lib/lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,816 copying build/lib/lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aclue 2026-05-11T15:53:05,818 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/webqs 2026-05-11T15:53:05,819 copying build/lib/lm_eval/tasks/webqs/webqs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/webqs 2026-05-11T15:53:05,821 copying build/lib/lm_eval/tasks/webqs/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/webqs 2026-05-11T15:53:05,824 copying build/lib/lm_eval/tasks/webqs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/webqs 2026-05-11T15:53:05,826 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,827 copying build/lib/lm_eval/tasks/hendrycks_ethics/justice.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,830 copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,832 copying build/lib/lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,834 copying build/lib/lm_eval/tasks/hendrycks_ethics/commonsense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,836 copying build/lib/lm_eval/tasks/hendrycks_ethics/virtue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,839 copying build/lib/lm_eval/tasks/hendrycks_ethics/deontology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,841 copying build/lib/lm_eval/tasks/hendrycks_ethics/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,843 copying build/lib/lm_eval/tasks/hendrycks_ethics/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/hendrycks_ethics 2026-05-11T15:53:05,846 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm 2026-05-11T15:53:05,848 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T15:53:05,850 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,851 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,854 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,857 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,859 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,862 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,864 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,866 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,869 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,871 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,873 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,875 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,877 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,879 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,881 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,884 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,886 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,888 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,890 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,892 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,895 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_2 2026-05-11T15:53:05,898 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,899 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,901 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,904 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,906 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,908 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,910 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,913 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,915 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,917 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,919 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,921 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,923 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,924 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,926 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,928 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,929 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,931 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,933 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,934 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,936 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_1 2026-05-11T15:53:05,939 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,940 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,942 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,944 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,946 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,948 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,950 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,951 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,953 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,955 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,957 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,959 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,961 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,963 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,965 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,966 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,968 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,970 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,972 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,974 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,975 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_5 2026-05-11T15:53:05,977 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/afrimgsm_cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot 2026-05-11T15:53:05,980 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,981 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,983 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,985 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,987 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,989 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,990 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,992 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,994 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,996 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,998 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:05,999 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,001 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,003 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,005 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,007 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,009 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,010 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,012 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,014 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,016 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_3 2026-05-11T15:53:06,018 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,019 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,021 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,023 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,025 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,027 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,029 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,031 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,033 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,035 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,036 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,038 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,040 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,042 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,044 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,045 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,047 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,049 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,051 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,053 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,054 copying build/lib/lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct_cot/prompt_4 2026-05-11T15:53:06,056 copying build/lib/lm_eval/tasks/afrimgsm/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T15:53:06,059 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate 2026-05-11T15:53:06,061 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,062 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,064 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,066 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,069 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,070 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,072 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,074 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,077 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,079 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,081 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,082 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,084 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,086 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,088 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,090 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,092 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,094 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,096 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_2 2026-05-11T15:53:06,099 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,100 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,102 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,104 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,106 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,108 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,110 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,112 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,114 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,115 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,117 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,119 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,121 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,123 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,125 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,127 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,129 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,131 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,132 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_1 2026-05-11T15:53:06,135 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,136 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,138 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,140 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,142 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,143 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,145 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,147 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,149 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,150 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,152 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,154 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,156 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,158 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,160 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,162 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,164 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,166 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,167 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_5 2026-05-11T15:53:06,169 copying build/lib/lm_eval/tasks/afrimgsm/translate/afrimgsm_tt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate 2026-05-11T15:53:06,173 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,174 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,176 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,177 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,179 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,181 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,183 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,185 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,186 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,188 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,190 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,192 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,194 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,196 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,198 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,200 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,202 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,204 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,206 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_3 2026-05-11T15:53:06,209 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,210 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,211 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,213 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,215 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,217 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,218 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,220 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,222 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,224 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,225 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,227 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,229 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,231 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,233 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,234 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,236 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,238 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,240 copying build/lib/lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate/prompt_4 2026-05-11T15:53:06,242 copying build/lib/lm_eval/tasks/afrimgsm/run.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T15:53:06,243 copying build/lib/lm_eval/tasks/afrimgsm/gen_yaml.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T15:53:06,245 copying build/lib/lm_eval/tasks/afrimgsm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T15:53:06,247 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T15:53:06,249 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,250 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,252 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,254 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,256 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,258 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,260 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,262 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,264 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,265 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,267 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,269 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,271 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,273 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,275 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,277 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,279 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,281 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,283 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,284 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_2 2026-05-11T15:53:06,287 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,288 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,290 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,291 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,293 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,295 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,296 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,298 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,300 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,302 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,303 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,305 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,307 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,309 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,311 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,312 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,314 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,316 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,318 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,320 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_1 2026-05-11T15:53:06,322 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,323 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,325 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,327 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,329 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,331 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,333 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,335 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,337 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,339 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,340 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,342 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,344 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,346 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,348 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,350 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,352 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,354 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,356 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,357 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_5 2026-05-11T15:53:06,359 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/afrimgsm_tt_cot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot 2026-05-11T15:53:06,362 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,363 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,365 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,367 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,369 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,370 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,372 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,374 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,376 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,377 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,379 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,381 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,383 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,385 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,387 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,389 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,391 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,393 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,395 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,396 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_3 2026-05-11T15:53:06,399 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,400 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,402 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,404 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,406 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,408 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,410 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,411 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,413 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,415 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,417 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,419 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,421 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,423 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,425 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,427 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,428 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,430 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,432 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,434 copying build/lib/lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/translate_cot/prompt_4 2026-05-11T15:53:06,436 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct 2026-05-11T15:53:06,438 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,439 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,441 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,443 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,445 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,446 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,448 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,450 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,452 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,454 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,455 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,457 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,459 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,461 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,463 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,465 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,467 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,468 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,470 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,472 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,474 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_2 2026-05-11T15:53:06,476 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,477 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,479 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,481 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,483 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,484 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,486 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,487 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,489 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,491 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,493 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,495 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,498 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,499 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,501 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,503 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,506 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,508 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,510 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,512 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,514 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_1 2026-05-11T15:53:06,516 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,518 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,520 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,522 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,524 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,526 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,528 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,530 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,531 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,533 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,535 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,537 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,539 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,541 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,543 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,545 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,547 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,549 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,551 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,553 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,555 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_5 2026-05-11T15:53:06,557 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,558 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,560 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,562 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,564 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,566 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,568 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,570 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,572 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,574 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,575 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,577 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,579 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,581 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,582 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,584 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,586 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,588 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,590 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,593 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,595 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_3 2026-05-11T15:53:06,597 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,598 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,600 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,602 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,604 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,606 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,608 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,610 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_vai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,612 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,614 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,616 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,618 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,620 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,622 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,624 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,625 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,628 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,629 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,631 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,633 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,635 copying build/lib/lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct/prompt_4 2026-05-11T15:53:06,637 copying build/lib/lm_eval/tasks/afrimgsm/direct/afrimgsm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm/direct 2026-05-11T15:53:06,639 copying build/lib/lm_eval/tasks/afrimgsm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrimgsm 2026-05-11T15:53:06,642 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/bertaqa 2026-05-11T15:53:06,643 copying build/lib/lm_eval/tasks/bertaqa/_bertaqa_template -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,645 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,647 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,649 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,651 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,653 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,655 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,657 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,659 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,661 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,663 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,665 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,667 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,669 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,671 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,672 copying build/lib/lm_eval/tasks/bertaqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,675 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,676 copying build/lib/lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/bertaqa 2026-05-11T15:53:06,679 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/unscramble 2026-05-11T15:53:06,680 copying build/lib/lm_eval/tasks/unscramble/cycle_letters.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T15:53:06,682 copying build/lib/lm_eval/tasks/unscramble/anagrams2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T15:53:06,684 copying build/lib/lm_eval/tasks/unscramble/reversed_words.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T15:53:06,686 copying build/lib/lm_eval/tasks/unscramble/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T15:53:06,688 copying build/lib/lm_eval/tasks/unscramble/random_insertion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T15:53:06,690 copying build/lib/lm_eval/tasks/unscramble/anagrams1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/unscramble 2026-05-11T15:53:06,693 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arc_mt 2026-05-11T15:53:06,694 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,696 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,698 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,700 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,702 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,704 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,706 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,708 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,710 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,712 copying build/lib/lm_eval/tasks/arc_mt/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,713 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,715 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,717 copying build/lib/lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arc_mt 2026-05-11T15:53:06,720 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openbookqa 2026-05-11T15:53:06,721 copying build/lib/lm_eval/tasks/openbookqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openbookqa 2026-05-11T15:53:06,723 copying build/lib/lm_eval/tasks/openbookqa/openbookqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openbookqa 2026-05-11T15:53:06,725 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/nq_open 2026-05-11T15:53:06,727 copying build/lib/lm_eval/tasks/nq_open/nq_open.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nq_open 2026-05-11T15:53:06,728 copying build/lib/lm_eval/tasks/nq_open/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/nq_open 2026-05-11T15:53:06,731 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xwinograd 2026-05-11T15:53:06,732 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,734 copying build/lib/lm_eval/tasks/xwinograd/_xwinograd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,736 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,738 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,739 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,741 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,743 copying build/lib/lm_eval/tasks/xwinograd/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,745 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_jp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,747 copying build/lib/lm_eval/tasks/xwinograd/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,749 copying build/lib/lm_eval/tasks/xwinograd/xwinograd_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xwinograd 2026-05-11T15:53:06,752 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/med_prescriptions 2026-05-11T15:53:06,753 copying build/lib/lm_eval/tasks/med_prescriptions/med_prescriptions_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_prescriptions 2026-05-11T15:53:06,755 copying build/lib/lm_eval/tasks/med_prescriptions/med_prescriptions_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_prescriptions 2026-05-11T15:53:06,757 copying build/lib/lm_eval/tasks/med_prescriptions/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_prescriptions 2026-05-11T15:53:06,761 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli 2026-05-11T15:53:06,762 copying build/lib/lm_eval/tasks/afrixnli/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli 2026-05-11T15:53:06,765 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate 2026-05-11T15:53:06,767 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,768 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,770 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,772 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,774 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,776 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,778 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,780 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,782 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,784 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,786 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,787 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,789 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,791 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,793 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,795 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,797 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,799 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,801 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,802 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_2 2026-05-11T15:53:06,805 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,806 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,808 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,810 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,811 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,813 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,815 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,817 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,819 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,821 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,823 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,825 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,826 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,828 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,830 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,832 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,833 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,835 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,837 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,839 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_1 2026-05-11T15:53:06,842 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,843 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,845 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,847 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,849 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,851 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,853 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,855 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,857 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,858 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,860 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,862 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,864 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,866 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,868 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,869 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,871 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,873 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,875 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,877 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_5 2026-05-11T15:53:06,879 copying build/lib/lm_eval/tasks/afrixnli/translate/afrixnli_tt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate 2026-05-11T15:53:06,881 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,882 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,884 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,886 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,888 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,890 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,892 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,893 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,895 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,897 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,899 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,901 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,903 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,905 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,907 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,909 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,911 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,913 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,914 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,916 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_3 2026-05-11T15:53:06,919 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,920 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,921 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,923 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,925 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,927 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,929 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,931 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,933 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,934 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,936 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,938 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,940 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,942 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,944 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,946 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,947 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,949 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,951 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,953 copying build/lib/lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/translate/prompt_4 2026-05-11T15:53:06,955 copying build/lib/lm_eval/tasks/afrixnli/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli 2026-05-11T15:53:06,957 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct 2026-05-11T15:53:06,959 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,960 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,962 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,964 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,966 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,967 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,969 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,971 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,973 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,975 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,977 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,979 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,981 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,982 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,984 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,986 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,988 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,990 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,991 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,993 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,995 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_2 2026-05-11T15:53:06,997 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:06,999 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,001 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,003 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,004 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,006 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,008 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,010 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,012 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,014 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,016 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,017 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,019 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,021 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,023 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,025 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,027 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,029 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,031 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,032 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,034 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_1 2026-05-11T15:53:07,037 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,038 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,040 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,042 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,043 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,045 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,047 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,049 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,051 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,052 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,054 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,056 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,059 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,061 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,062 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,064 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,066 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,068 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,070 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,072 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,074 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_5 2026-05-11T15:53:07,076 copying build/lib/lm_eval/tasks/afrixnli/direct/afrixnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct 2026-05-11T15:53:07,078 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,079 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,081 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,083 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,085 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,087 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,089 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,091 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,093 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,095 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,097 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,099 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,101 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,102 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,104 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,106 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,108 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,110 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,112 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,114 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,116 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_3 2026-05-11T15:53:07,119 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,120 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,122 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,124 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,126 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,128 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,130 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,131 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,133 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,135 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,137 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,139 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,141 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,143 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,144 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,146 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,148 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,150 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,152 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,154 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,156 copying build/lib/lm_eval/tasks/afrixnli/direct/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/direct/prompt_4 2026-05-11T15:53:07,158 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/lai prompt 2026-05-11T15:53:07,160 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,161 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,163 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,165 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,167 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,169 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,171 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,173 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,175 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,177 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,179 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,181 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,182 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,184 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,186 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,188 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,190 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,192 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,194 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,196 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/translate 2026-05-11T15:53:07,199 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,200 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,202 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,205 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,207 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,209 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,210 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,212 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,214 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,216 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,218 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,220 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,222 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,223 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,225 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,227 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,229 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,231 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,233 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,235 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,236 copying build/lib/lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/lai prompt/direct 2026-05-11T15:53:07,238 copying build/lib/lm_eval/tasks/afrixnli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli 2026-05-11T15:53:07,241 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt 2026-05-11T15:53:07,243 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,244 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,246 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,248 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,250 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,252 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,254 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,255 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,257 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,259 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,260 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,262 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,264 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,266 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,269 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,271 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,272 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,274 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,276 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,278 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,280 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/native-direct 2026-05-11T15:53:07,283 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,284 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,286 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,288 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,290 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,291 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,293 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,295 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,297 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,298 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,300 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,302 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,304 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,305 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,307 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,309 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,311 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,313 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,315 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,317 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/translate 2026-05-11T15:53:07,319 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,320 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,323 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,325 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,327 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,329 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,331 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,333 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,335 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,337 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,340 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,342 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,344 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,346 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,349 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,351 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,353 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,355 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,358 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,360 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,362 copying build/lib/lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrixnli/anli prompt/en-direct 2026-05-11T15:53:07,365 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/siqa 2026-05-11T15:53:07,366 copying build/lib/lm_eval/tasks/siqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/siqa 2026-05-11T15:53:07,369 copying build/lib/lm_eval/tasks/siqa/siqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/siqa 2026-05-11T15:53:07,373 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/agieval 2026-05-11T15:53:07,374 copying build/lib/lm_eval/tasks/agieval/logiqa-zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,377 copying build/lib/lm_eval/tasks/agieval/agieval_nous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,379 copying build/lib/lm_eval/tasks/agieval/gaokao-chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,382 copying build/lib/lm_eval/tasks/agieval/agieval_cn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,384 copying build/lib/lm_eval/tasks/agieval/gaokao-chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,386 copying build/lib/lm_eval/tasks/agieval/aqua-rat.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,389 copying build/lib/lm_eval/tasks/agieval/jec-qa-kd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,391 copying build/lib/lm_eval/tasks/agieval/gaokao-history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,393 copying build/lib/lm_eval/tasks/agieval/gaokao-english.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,396 copying build/lib/lm_eval/tasks/agieval/gaokao-physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,398 copying build/lib/lm_eval/tasks/agieval/math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,400 copying build/lib/lm_eval/tasks/agieval/lsat-lr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,402 copying build/lib/lm_eval/tasks/agieval/sat-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,405 copying build/lib/lm_eval/tasks/agieval/gaokao-mathqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,407 copying build/lib/lm_eval/tasks/agieval/sat-math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,409 copying build/lib/lm_eval/tasks/agieval/sat-en-without-passage.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,412 copying build/lib/lm_eval/tasks/agieval/jec-qa-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,414 copying build/lib/lm_eval/tasks/agieval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,417 copying build/lib/lm_eval/tasks/agieval/gaokao-mathcloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,419 copying build/lib/lm_eval/tasks/agieval/agieval_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,421 copying build/lib/lm_eval/tasks/agieval/agieval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,424 copying build/lib/lm_eval/tasks/agieval/gaokao-biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,426 copying build/lib/lm_eval/tasks/agieval/logiqa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,428 copying build/lib/lm_eval/tasks/agieval/gaokao-geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,431 copying build/lib/lm_eval/tasks/agieval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,433 copying build/lib/lm_eval/tasks/agieval/lsat-ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,435 copying build/lib/lm_eval/tasks/agieval/lsat-rc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/agieval 2026-05-11T15:53:07,437 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/babi 2026-05-11T15:53:07,438 copying build/lib/lm_eval/tasks/babi/babi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babi 2026-05-11T15:53:07,441 copying build/lib/lm_eval/tasks/babi/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babi 2026-05-11T15:53:07,443 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval 2026-05-11T15:53:07,444 copying build/lib/lm_eval/tasks/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T15:53:07,447 copying build/lib/lm_eval/tasks/ifeval/instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T15:53:07,449 copying build/lib/lm_eval/tasks/ifeval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T15:53:07,451 copying build/lib/lm_eval/tasks/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T15:53:07,454 copying build/lib/lm_eval/tasks/ifeval/ifeval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T15:53:07,457 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval/multilingual 2026-05-11T15:53:07,458 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T15:53:07,461 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:53:07,462 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:53:07,464 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions/es_instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:53:07,467 copying build/lib/lm_eval/tasks/ifeval/multilingual/instructions/ca_instructions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instructions 2026-05-11T15:53:07,471 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:53:07,472 copying build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils/es_instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:53:07,474 copying build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:53:07,476 copying build/lib/lm_eval/tasks/ifeval/multilingual/instruction_utils/ca_instructions_util.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual/instruction_utils 2026-05-11T15:53:07,478 copying build/lib/lm_eval/tasks/ifeval/multilingual/ifeval_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T15:53:07,480 copying build/lib/lm_eval/tasks/ifeval/multilingual/ifeval_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T15:53:07,482 copying build/lib/lm_eval/tasks/ifeval/multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval/multilingual 2026-05-11T15:53:07,484 copying build/lib/lm_eval/tasks/ifeval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/ifeval 2026-05-11T15:53:07,487 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm 2026-05-11T15:53:07,489 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,490 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,492 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,494 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,496 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,498 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,500 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,502 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,504 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,506 copying build/lib/lm_eval/tasks/mgsm/en_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,508 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,510 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,512 copying build/lib/lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/en_cot 2026-05-11T15:53:07,513 copying build/lib/lm_eval/tasks/mgsm/gen_yaml.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm 2026-05-11T15:53:07,515 copying build/lib/lm_eval/tasks/mgsm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm 2026-05-11T15:53:07,518 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,519 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,521 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,523 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,525 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,527 copying build/lib/lm_eval/tasks/mgsm/native_cot/cot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,529 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,531 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,533 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,535 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,537 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,539 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,540 copying build/lib/lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/native_cot 2026-05-11T15:53:07,543 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,544 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,546 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,548 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,549 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,551 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,553 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,555 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,557 copying build/lib/lm_eval/tasks/mgsm/direct/direct_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,559 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,561 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,564 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,566 copying build/lib/lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm/direct 2026-05-11T15:53:07,569 copying build/lib/lm_eval/tasks/mgsm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mgsm 2026-05-11T15:53:07,572 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/asdiv 2026-05-11T15:53:07,573 copying build/lib/lm_eval/tasks/asdiv/asdiv-cot-llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/asdiv 2026-05-11T15:53:07,576 copying build/lib/lm_eval/tasks/asdiv/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/asdiv 2026-05-11T15:53:07,578 copying build/lib/lm_eval/tasks/asdiv/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/asdiv 2026-05-11T15:53:07,582 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lambada 2026-05-11T15:53:07,584 copying build/lib/lm_eval/tasks/lambada/lambada_openai.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada 2026-05-11T15:53:07,587 copying build/lib/lm_eval/tasks/lambada/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada 2026-05-11T15:53:07,589 copying build/lib/lm_eval/tasks/lambada/lambada_standard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lambada 2026-05-11T15:53:07,592 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/triviaqa 2026-05-11T15:53:07,593 copying build/lib/lm_eval/tasks/triviaqa/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/triviaqa 2026-05-11T15:53:07,595 copying build/lib/lm_eval/tasks/triviaqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/triviaqa 2026-05-11T15:53:07,597 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,598 copying build/lib/lm_eval/tasks/tinyBenchmarks/agg_functions.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,601 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,602 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,605 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,606 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,608 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,611 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,613 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,615 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,616 copying build/lib/lm_eval/tasks/tinyBenchmarks/utils_winogrande.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,618 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,620 copying build/lib/lm_eval/tasks/tinyBenchmarks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,622 copying build/lib/lm_eval/tasks/tinyBenchmarks/tinyArc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tinyBenchmarks 2026-05-11T15:53:07,625 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,626 copying build/lib/lm_eval/tasks/humaneval_infilling/single_line_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,628 copying build/lib/lm_eval/tasks/humaneval_infilling/random_span_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,630 copying build/lib/lm_eval/tasks/humaneval_infilling/multi_line_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,632 copying build/lib/lm_eval/tasks/humaneval_infilling/random_span_infilling_light.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,633 copying build/lib/lm_eval/tasks/humaneval_infilling/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,635 copying build/lib/lm_eval/tasks/humaneval_infilling/humaneval_infilling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,637 copying build/lib/lm_eval/tasks/humaneval_infilling/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/humaneval_infilling 2026-05-11T15:53:07,640 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,641 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,644 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_history_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,645 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,647 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_philosophy_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,649 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,651 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_geography_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,652 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_economics_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,654 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_geography_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,656 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_political_science_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,658 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,660 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_economics_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,662 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,664 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,665 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,667 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,669 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_law_professional.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,671 copying build/lib/lm_eval/tasks/arabicmmlu/_default_arabicmmlu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,672 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_management_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,674 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,675 copying build/lib/lm_eval/tasks/arabicmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,677 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,679 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,681 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_history_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,683 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_history_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,684 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_civics_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,686 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,688 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_math_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,689 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,691 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_geography_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,693 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_physics_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,695 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_economics_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,696 copying build/lib/lm_eval/tasks/arabicmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,698 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,700 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,702 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,704 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_accounting_university.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,705 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,707 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,709 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_civics_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,711 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,713 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,714 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_primary_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,716 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,718 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,719 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_biology_high_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,721 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,723 copying build/lib/lm_eval/tasks/arabicmmlu/_arabicmmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,725 copying build/lib/lm_eval/tasks/arabicmmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,726 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_middle_school.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,728 copying build/lib/lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arabicmmlu 2026-05-11T15:53:07,731 copying build/lib/lm_eval/tasks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T15:53:07,737 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/french_bench 2026-05-11T15:53:07,738 copying build/lib/lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,741 copying build/lib/lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,743 copying build/lib/lm_eval/tasks/french_bench/french_bench_boolqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,745 copying build/lib/lm_eval/tasks/french_bench/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,747 copying build/lib/lm_eval/tasks/french_bench/french_bench_xnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,750 copying build/lib/lm_eval/tasks/french_bench/french_bench_multifquad.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,752 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,754 copying build/lib/lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,756 copying build/lib/lm_eval/tasks/french_bench/french_bench_grammar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,759 copying build/lib/lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,761 copying build/lib/lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,763 copying build/lib/lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,765 copying build/lib/lm_eval/tasks/french_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,768 copying build/lib/lm_eval/tasks/french_bench/french_bench_reading_comp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,770 copying build/lib/lm_eval/tasks/french_bench/preprocess_wikitext.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,773 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,775 copying build/lib/lm_eval/tasks/french_bench/french_bench_trivia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,777 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,780 copying build/lib/lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,783 copying build/lib/lm_eval/tasks/french_bench/french_bench_vocab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,785 copying build/lib/lm_eval/tasks/french_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,787 copying build/lib/lm_eval/tasks/french_bench/french_bench_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/french_bench 2026-05-11T15:53:07,792 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,794 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,796 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,798 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,801 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,803 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,805 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__sing_MS_MV_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,807 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,809 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,811 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,814 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,816 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,818 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,821 copying build/lib/lm_eval/tasks/lm_syneval/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,823 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,825 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,827 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,830 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,832 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_plur_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,834 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,836 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,839 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,841 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,843 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,845 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__plur_MS_ANPHR.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,848 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,851 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,853 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,855 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,857 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__plur_MS_MV_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,859 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,861 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,864 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,866 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,868 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,870 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,873 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,875 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,878 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,880 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,883 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,885 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,888 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,890 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,893 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,895 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_sing_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,898 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,900 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,902 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,904 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,906 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,909 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,911 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__plur_MS_LMV_LMV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,914 copying build/lib/lm_eval/tasks/lm_syneval/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,918 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__future.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,920 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,922 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__sing_MS_LMV_LMV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,924 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_sing_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,926 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,928 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_sing_BS.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,930 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,931 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,933 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,935 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,937 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,939 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,941 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_sing_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,943 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__sing_MS_ANPHR.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,945 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,947 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_plur_MS_MV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,949 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_plur_ES.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,951 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_sing_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,953 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_plur_ES_EV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,955 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,957 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_plur_IS_IV.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,959 copying build/lib/lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__past.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/lm_syneval 2026-05-11T15:53:07,962 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/egymmlu 2026-05-11T15:53:07,963 copying build/lib/lm_eval/tasks/egymmlu/_egymmlu_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,965 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,967 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_driving_test.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,969 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,971 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,973 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,974 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,976 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_islamic_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,978 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_arabic_language_(general).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,979 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,981 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,983 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,985 copying build/lib/lm_eval/tasks/egymmlu/_default_egymmlu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,987 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,989 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,990 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,992 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,994 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,996 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_philosophy_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,997 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:07,999 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_arabic_language_(grammar).yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,001 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,003 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,004 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,006 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_political_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,008 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,010 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,012 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,013 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_natural_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,015 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,017 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,018 copying build/lib/lm_eval/tasks/egymmlu/_egymmlu_ar_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,020 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,022 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_arabic_language.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,023 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,025 copying build/lib/lm_eval/tasks/egymmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,027 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,029 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_management_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,031 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,033 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,035 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,036 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,038 copying build/lib/lm_eval/tasks/egymmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,040 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,042 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,044 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,046 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,048 copying build/lib/lm_eval/tasks/egymmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,050 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,052 copying build/lib/lm_eval/tasks/egymmlu/_egymmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,054 copying build/lib/lm_eval/tasks/egymmlu/egymmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egymmlu 2026-05-11T15:53:08,056 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu 2026-05-11T15:53:08,057 copying build/lib/lm_eval/tasks/global_mmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu 2026-05-11T15:53:08,060 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default 2026-05-11T15:53:08,062 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,063 copying build/lib/lm_eval/tasks/global_mmlu/default/de/_de_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,065 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,067 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,068 copying build/lib/lm_eval/tasks/global_mmlu/default/de/_global_mmlu_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,070 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,072 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,074 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,075 copying build/lib/lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,077 copying build/lib/lm_eval/tasks/global_mmlu/default/de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/de 2026-05-11T15:53:08,080 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,080 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,082 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,084 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,086 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,088 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,090 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/_global_mmlu_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,092 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,094 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,096 copying build/lib/lm_eval/tasks/global_mmlu/default/yo/_yo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/yo 2026-05-11T15:53:08,099 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,100 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,102 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,104 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,106 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,108 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,110 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,112 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,114 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,116 copying build/lib/lm_eval/tasks/global_mmlu/default/zh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/zh 2026-05-11T15:53:08,118 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,119 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,121 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,123 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,125 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,127 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,129 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,131 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,133 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,134 copying build/lib/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ko 2026-05-11T15:53:08,138 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,139 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/_global_mmlu_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,142 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/_bn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,144 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,146 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,149 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,151 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,153 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,155 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,158 copying build/lib/lm_eval/tasks/global_mmlu/default/bn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/bn 2026-05-11T15:53:08,161 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,162 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,165 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,167 copying build/lib/lm_eval/tasks/global_mmlu/default/en/_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,169 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,171 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,174 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,176 copying build/lib/lm_eval/tasks/global_mmlu/default/en/_global_mmlu_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,178 copying build/lib/lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,180 copying build/lib/lm_eval/tasks/global_mmlu/default/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/en 2026-05-11T15:53:08,183 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,184 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,187 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,189 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,191 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/_global_mmlu_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,194 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,196 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,199 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,202 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,205 copying build/lib/lm_eval/tasks/global_mmlu/default/sw/_sw_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/sw 2026-05-11T15:53:08,208 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,209 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,212 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,214 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,216 copying build/lib/lm_eval/tasks/global_mmlu/default/es/_es_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,218 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,220 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,222 copying build/lib/lm_eval/tasks/global_mmlu/default/es/_global_mmlu_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,225 copying build/lib/lm_eval/tasks/global_mmlu/default/es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,227 copying build/lib/lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/es 2026-05-11T15:53:08,231 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,232 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/_ja_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,235 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,237 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,239 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,241 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,243 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/_global_mmlu_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,245 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,247 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,249 copying build/lib/lm_eval/tasks/global_mmlu/default/ja/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ja 2026-05-11T15:53:08,252 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,253 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,256 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/_hi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,258 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,260 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,263 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,265 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,267 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,269 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/_global_mmlu_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,272 copying build/lib/lm_eval/tasks/global_mmlu/default/hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/hi 2026-05-11T15:53:08,275 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,276 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,278 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,281 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,283 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,286 copying build/lib/lm_eval/tasks/global_mmlu/default/it/_it_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,288 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,290 copying build/lib/lm_eval/tasks/global_mmlu/default/it/_global_mmlu_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,293 copying build/lib/lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,295 copying build/lib/lm_eval/tasks/global_mmlu/default/it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/it 2026-05-11T15:53:08,298 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,299 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,302 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,304 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/_fr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,307 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,309 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,312 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/_global_mmlu_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,314 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,316 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,319 copying build/lib/lm_eval/tasks/global_mmlu/default/fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/fr 2026-05-11T15:53:08,322 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,323 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,325 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,327 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,329 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,331 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,333 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,334 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,336 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,338 copying build/lib/lm_eval/tasks/global_mmlu/default/pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/pt 2026-05-11T15:53:08,341 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,342 copying build/lib/lm_eval/tasks/global_mmlu/default/id/_id_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,344 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,346 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,348 copying build/lib/lm_eval/tasks/global_mmlu/default/id/_global_mmlu_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,350 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,351 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,353 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,355 copying build/lib/lm_eval/tasks/global_mmlu/default/id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,357 copying build/lib/lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/id 2026-05-11T15:53:08,359 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,361 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,363 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/_ar_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,364 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,366 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/_global_mmlu_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,368 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,370 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,372 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,374 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,375 copying build/lib/lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_medical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/default/ar 2026-05-11T15:53:08,378 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full 2026-05-11T15:53:08,381 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,382 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,384 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,386 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,388 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,390 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,392 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,394 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,396 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_ha_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,398 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,400 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,402 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,404 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,406 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,408 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,410 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,412 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,414 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,415 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,417 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,419 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,421 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,423 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,424 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,426 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,429 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,431 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,433 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,435 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,436 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,438 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,440 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,442 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,444 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,446 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,448 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,450 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,452 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,454 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,456 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,457 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,459 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,461 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,463 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,465 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,467 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,469 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,471 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,472 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,474 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,476 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,478 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,480 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,482 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,484 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,486 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,488 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,490 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,492 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,494 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,495 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,497 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,500 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,501 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,503 copying build/lib/lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ha 2026-05-11T15:53:08,507 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,508 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,510 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,513 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_ky_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,514 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,516 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,519 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,521 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,522 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,524 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,526 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,528 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,530 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,532 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,534 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,536 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,538 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,540 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,542 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,544 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,545 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,547 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,549 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,552 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,554 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,557 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,559 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,561 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,563 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,565 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,566 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,568 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,570 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,572 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,574 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,576 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,578 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,580 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,583 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,585 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,587 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,589 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,591 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,593 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,595 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,598 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,600 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,602 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,605 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,607 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,610 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,612 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,615 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,617 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,620 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,622 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,624 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,626 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,628 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,630 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,632 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,634 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,636 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,637 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,639 copying build/lib/lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ky 2026-05-11T15:53:08,643 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,644 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,646 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,648 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,649 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,651 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,653 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_de_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,655 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,657 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,659 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,661 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,663 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,665 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,667 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,669 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,671 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,672 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,674 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,676 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,678 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,680 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,682 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,684 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,686 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,687 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,689 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,691 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,693 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,695 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,697 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,699 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,701 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,702 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,704 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,706 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,708 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,710 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,712 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,714 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,716 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,718 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,720 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,722 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,723 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,725 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,727 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,729 copying build/lib/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,731 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,733 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,735 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,737 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,739 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,741 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,742 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,744 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,746 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,748 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,750 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,752 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,754 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,756 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,758 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,760 copying build/lib/lm_eval/tasks/global_mmlu/full/de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,762 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,764 copying build/lib/lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/de 2026-05-11T15:53:08,768 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,769 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,771 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,773 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,775 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,777 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,779 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,780 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,782 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,784 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,786 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,788 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,790 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,792 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,794 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,795 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,797 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,799 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,801 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,803 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,805 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,807 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,809 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,810 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,812 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,814 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,816 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,818 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,820 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,822 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,824 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,826 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,828 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,830 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,832 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,833 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,835 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,837 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,839 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,841 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,844 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,846 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,848 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,850 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,852 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_so_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,854 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,856 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,858 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,860 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,862 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,864 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,866 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,868 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,870 copying build/lib/lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,872 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,875 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,877 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,879 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,881 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,883 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,885 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,887 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,889 copying build/lib/lm_eval/tasks/global_mmlu/full/so/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,892 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,894 copying build/lib/lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/so 2026-05-11T15:53:08,899 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,900 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,903 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,905 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,907 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,910 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,912 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,914 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,917 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,919 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,921 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,923 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,925 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,927 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,929 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,931 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,933 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,935 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,937 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,938 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,940 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,942 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,944 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,947 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,949 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,951 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,954 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,956 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,958 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,961 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,963 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,965 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,968 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,970 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,972 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,974 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,976 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,979 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,981 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,983 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,986 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,989 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,991 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,993 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,996 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:08,998 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,000 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,002 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,005 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,007 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,009 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,012 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,014 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,016 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,019 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,021 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,023 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,026 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,028 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,030 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,033 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,035 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,038 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,041 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/_yo_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,043 copying build/lib/lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/yo 2026-05-11T15:53:09,046 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,048 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,050 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,052 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,054 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,055 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,057 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,059 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,061 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,063 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,065 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,067 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,069 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,071 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,073 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,075 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,077 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,079 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,081 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,083 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,085 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,087 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,088 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,090 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,092 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,094 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,096 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,097 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,099 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,101 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,103 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,105 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,107 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,109 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,111 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,113 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,115 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,117 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,119 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,121 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,123 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,125 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,127 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,129 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,136 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,138 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,140 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,142 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,143 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,145 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,147 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,149 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,151 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/_ru_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,153 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,155 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,157 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,159 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,161 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,163 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,165 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,167 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,169 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,171 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,173 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,175 copying build/lib/lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ru 2026-05-11T15:53:09,179 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,180 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,182 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,184 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,186 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,187 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,189 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,191 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,193 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,195 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,197 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,199 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,201 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_zh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,203 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,205 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,207 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,209 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,211 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,213 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,214 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,216 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,218 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,220 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,222 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,224 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,225 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,227 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,229 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,231 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,233 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,235 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,237 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,238 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,240 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,242 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,244 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,246 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,248 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,250 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,252 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,254 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,256 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,258 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,260 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,262 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,264 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,265 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,267 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,269 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,271 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,273 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,275 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,277 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,279 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,281 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,283 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,285 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,286 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,289 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,290 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,292 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,294 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,296 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,299 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,301 copying build/lib/lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/zh 2026-05-11T15:53:09,304 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,306 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,308 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,309 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,311 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,313 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,315 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,317 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,319 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,320 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,322 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,324 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,326 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,328 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,330 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,331 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,333 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,335 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,337 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,339 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,341 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,343 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,345 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,347 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,348 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,350 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,352 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,354 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,356 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,357 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_nl_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,359 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,361 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,363 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,365 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,366 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,368 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,370 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,372 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,374 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,376 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,377 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,379 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,381 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,383 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,385 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,387 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,388 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,390 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,392 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,394 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,396 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,397 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,400 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,402 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,403 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,405 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,407 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,409 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,411 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,413 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,415 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,417 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,419 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,421 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,423 copying build/lib/lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/nl 2026-05-11T15:53:09,426 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,427 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,429 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,431 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,433 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,435 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,437 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,439 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,441 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,443 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,445 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,447 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,449 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,450 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,452 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,454 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,456 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,458 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,460 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,462 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,464 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,466 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,468 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,470 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,471 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,473 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,475 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,477 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,479 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,481 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,483 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,485 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,487 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,489 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,491 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,493 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,495 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,497 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,499 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,501 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,503 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,505 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,507 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,509 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,511 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,513 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,515 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,517 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,519 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,521 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,522 copying build/lib/lm_eval/tasks/global_mmlu/full/si/_si_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,524 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,526 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,528 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,529 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,531 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,533 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,535 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,537 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,538 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,540 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,547 copying build/lib/lm_eval/tasks/global_mmlu/full/si/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,549 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,551 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,553 copying build/lib/lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/si 2026-05-11T15:53:09,557 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,558 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,560 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,562 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,563 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,565 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,567 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,569 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,571 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,573 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,575 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,577 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,579 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,581 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,583 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,585 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,587 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,589 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,591 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,593 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,595 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,597 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,598 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,600 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,602 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,604 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,606 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,608 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,610 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,611 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,613 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,615 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,617 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,619 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,621 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,623 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,626 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,628 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,630 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,632 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,634 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,636 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_ne_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,638 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,640 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,642 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,644 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,645 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,648 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,650 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,653 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,656 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,658 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,661 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,663 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,666 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,669 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,671 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,673 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,675 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,677 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,680 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,682 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,684 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,686 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,688 copying build/lib/lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ne 2026-05-11T15:53:09,691 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,693 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,695 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,697 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,698 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,700 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_vi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,702 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,704 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,706 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,708 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,710 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,712 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,714 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,716 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,718 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,720 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,722 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,724 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,726 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,728 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,730 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,732 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,734 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,735 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,737 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,739 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,741 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,742 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,744 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,746 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,747 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,750 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,752 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,754 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,756 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,759 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,761 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,763 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,766 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,768 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,770 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,773 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,775 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,778 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,780 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,782 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,784 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,786 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,789 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,791 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,793 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,795 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,797 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,800 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,802 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,804 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,807 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,809 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,811 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,814 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,816 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,818 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,821 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,823 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,825 copying build/lib/lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/vi 2026-05-11T15:53:09,830 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,831 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,833 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,835 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,838 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,840 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,842 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,844 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,846 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,848 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,850 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,851 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,853 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,855 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,856 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,858 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,860 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,862 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,863 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,865 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,867 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,869 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,870 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,872 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,874 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,875 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,877 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,878 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,880 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,882 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,883 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,885 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,887 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,889 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,890 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,892 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,894 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,895 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,897 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,899 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,900 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,902 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,904 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,905 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,907 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,909 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,910 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,912 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,914 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,915 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,917 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,919 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,921 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,922 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,924 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/_ig_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,926 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,928 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,930 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,931 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,933 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,935 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,937 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,939 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,941 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,942 copying build/lib/lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ig 2026-05-11T15:53:09,946 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,947 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,949 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_ro_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,951 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,952 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,954 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,956 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,957 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,959 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,961 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,963 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,964 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,966 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,968 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,970 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,972 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,974 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,975 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,977 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,979 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,981 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,983 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,985 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,987 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,988 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,990 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,992 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,993 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,995 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,997 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:09,999 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,001 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,003 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,005 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,006 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,009 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,011 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,012 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,014 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,016 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,018 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,020 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,022 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,024 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,026 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,027 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,029 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,031 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,033 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,034 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,036 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,038 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,040 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,042 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,044 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,045 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,047 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,049 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,051 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,053 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,055 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,057 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,059 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,061 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,063 copying build/lib/lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ro 2026-05-11T15:53:10,067 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,067 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,069 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,071 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,073 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,075 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,077 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_ko_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,079 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,080 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,082 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,084 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,086 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,088 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,090 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,092 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,094 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,096 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,098 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,100 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,102 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,104 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,106 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,108 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,110 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,112 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,114 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,115 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,117 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,119 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,121 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,123 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,125 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,127 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,129 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,131 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,133 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,135 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,137 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,139 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,141 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,144 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,146 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,148 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,150 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,152 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,154 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,156 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,158 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,160 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,162 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,163 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,165 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,167 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,169 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,171 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,173 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,175 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,178 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,179 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,181 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,183 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,185 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,188 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,190 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,192 copying build/lib/lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ko 2026-05-11T15:53:10,196 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,197 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,199 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,201 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,203 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,205 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,207 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,209 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,210 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,212 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,214 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,216 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,218 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,220 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,222 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,223 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,225 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,227 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,229 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,231 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,233 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,235 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,237 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_lt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,239 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,241 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,243 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,245 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,247 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,249 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,250 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,252 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,254 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,256 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,258 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,260 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,262 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,264 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,265 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,267 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,269 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,271 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,273 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,275 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,277 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,279 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,280 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,282 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,285 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,287 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,289 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,291 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,292 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,294 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,296 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,298 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,300 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,302 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,304 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,306 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,308 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,310 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,312 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,314 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,316 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,318 copying build/lib/lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/lt 2026-05-11T15:53:10,321 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,322 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,324 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,326 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,327 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,329 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,331 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,333 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,334 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,336 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,338 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,340 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,342 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,343 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,345 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,347 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,349 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,350 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,352 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,354 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,356 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,358 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,360 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,362 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,363 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,365 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,366 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,368 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,370 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,372 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,373 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,375 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,377 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,379 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,381 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,382 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,384 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,386 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,388 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,390 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_pl_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,392 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,393 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,396 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,398 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,400 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,402 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,404 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,406 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,407 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,409 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,411 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,413 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,414 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,416 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,418 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,420 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,422 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,424 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,425 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,427 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,429 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,431 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,433 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,435 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,437 copying build/lib/lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pl 2026-05-11T15:53:10,440 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,442 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,444 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,445 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,447 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,449 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,451 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,453 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,455 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,457 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,459 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,460 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,462 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,464 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,466 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,468 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,470 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,472 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,474 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,476 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,478 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,480 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_sn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,482 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,484 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,486 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,487 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,489 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,491 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,493 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,495 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,497 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,499 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,501 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,503 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,505 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,507 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,509 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,510 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,512 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,514 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,516 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,518 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,519 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,521 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,523 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,525 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,527 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,528 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,530 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,532 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,533 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,535 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,537 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,539 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,541 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,543 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,544 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,546 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,548 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,550 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,552 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,554 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,557 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,559 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,561 copying build/lib/lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sn 2026-05-11T15:53:10,566 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,567 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,569 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,571 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,574 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_bn_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,576 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,578 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,580 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,582 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,584 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,586 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,589 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,591 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,593 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,595 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,598 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,600 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,602 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,604 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,607 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,609 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,611 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,614 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,616 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,618 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,620 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,622 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,624 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,627 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,629 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,631 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,633 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,635 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,637 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,639 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,642 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,644 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,646 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,648 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,650 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,653 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,654 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,656 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,658 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,660 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,662 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,664 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,665 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,667 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,669 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,671 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,673 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,674 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,676 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,678 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,680 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,682 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,684 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,686 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,688 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,690 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,692 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,693 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,695 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,697 copying build/lib/lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/bn 2026-05-11T15:53:10,701 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,702 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,704 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,706 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,708 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,710 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,712 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,713 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,715 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,717 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,719 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,721 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,723 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,725 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,727 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,729 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,731 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,733 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,735 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,737 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,738 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,740 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,742 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,744 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,746 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,748 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,750 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,752 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,754 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,755 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,757 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,759 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,761 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,762 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,764 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,766 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,768 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,770 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,772 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,774 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,776 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,778 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,780 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,782 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,784 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,786 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,789 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,791 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,793 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,795 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,797 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,798 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,800 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,802 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/_ny_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,804 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,806 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,808 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,810 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,812 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,814 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,816 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,818 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,820 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,822 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,824 copying build/lib/lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ny 2026-05-11T15:53:10,828 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,829 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,831 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,833 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,835 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,837 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,838 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,840 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,842 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,844 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,845 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,847 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,849 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,851 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,853 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,855 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,865 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,866 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,869 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,871 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,873 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,875 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,876 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,878 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,880 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,882 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,884 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,886 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,888 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,890 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,892 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,893 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,895 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,897 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,899 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,901 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,902 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,904 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,906 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,908 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,910 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,912 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,914 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,916 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,918 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,919 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,921 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,923 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,925 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,927 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,929 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,931 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,933 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,935 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,937 copying build/lib/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,939 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,941 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,943 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,945 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,947 copying build/lib/lm_eval/tasks/global_mmlu/full/en/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,949 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,952 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,954 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,956 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,958 copying build/lib/lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/en 2026-05-11T15:53:10,961 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,962 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,965 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,967 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,968 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,970 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_uk_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,972 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,974 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,976 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,978 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,980 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,982 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,984 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,986 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,988 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,990 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,991 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,994 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,996 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:10,998 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,000 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,002 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,004 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,006 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,008 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,010 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,012 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,014 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,016 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,018 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,020 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,022 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,023 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,026 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,027 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,029 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,031 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,033 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,035 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,037 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,039 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,041 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,043 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,045 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,047 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,049 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,051 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,053 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,055 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,057 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,059 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,061 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,063 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,065 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,067 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,069 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,071 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,073 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,075 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,077 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,079 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,081 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,083 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,085 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,087 copying build/lib/lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/uk 2026-05-11T15:53:11,091 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,092 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,094 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,096 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,098 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,100 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,101 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,103 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,105 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,107 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,109 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,110 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,112 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,114 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,116 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,117 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,119 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,121 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,123 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,125 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,127 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,129 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,130 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,133 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,135 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,137 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,139 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,141 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,142 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,144 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,146 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,148 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,150 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,152 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,154 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,156 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,158 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,160 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,162 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,164 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,166 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,167 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,169 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,171 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,173 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,175 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,177 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,179 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,181 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,183 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,185 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,186 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,188 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,190 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,192 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,194 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,196 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,198 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,200 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,202 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,204 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/_sw_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,206 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,208 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,210 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,212 copying build/lib/lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sw 2026-05-11T15:53:11,215 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,217 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,219 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,221 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,223 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,225 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,226 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,228 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,230 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,232 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,234 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,235 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,237 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,239 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,241 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,243 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,245 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,247 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,249 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,251 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,253 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,255 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,257 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,258 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,260 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,262 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,264 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,266 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_es_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,268 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,270 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,272 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,274 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,275 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,277 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,279 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,281 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,283 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,285 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,287 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,288 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,290 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,292 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,294 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,296 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,298 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,300 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,302 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,304 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,306 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,307 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,309 copying build/lib/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,311 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,313 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,315 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,317 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,319 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,321 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,322 copying build/lib/lm_eval/tasks/global_mmlu/full/es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,324 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,326 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,328 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,330 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,332 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,334 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,335 copying build/lib/lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/es 2026-05-11T15:53:11,339 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,340 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,342 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,344 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,346 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,348 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,350 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,351 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,353 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,355 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,357 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,359 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,361 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,364 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,366 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,368 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,370 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,373 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,375 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,377 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,380 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,382 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,385 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,387 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,389 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,392 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,394 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,396 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,399 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,401 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,404 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,406 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,408 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,411 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,413 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,415 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,417 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,420 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_te_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,422 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,424 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,427 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,429 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,432 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,434 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,436 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,438 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,441 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,443 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,446 copying build/lib/lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,448 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,450 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,452 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,455 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,457 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,459 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,461 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,463 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,465 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,467 copying build/lib/lm_eval/tasks/global_mmlu/full/te/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,469 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,471 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,473 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,475 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,477 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,479 copying build/lib/lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/te 2026-05-11T15:53:11,482 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,483 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,486 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,487 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,489 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,491 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,493 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,495 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,497 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,499 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,500 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,502 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,504 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,506 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,508 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,510 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,512 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,514 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,515 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,517 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,519 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,521 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,523 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,525 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,527 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_fil_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,529 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,531 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,533 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,535 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,537 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,539 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,540 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,542 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,544 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,546 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,547 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,549 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,551 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,553 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,555 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,557 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,559 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,561 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,563 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,564 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,566 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,568 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,570 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,572 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,574 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,575 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,577 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,579 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,581 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,582 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,584 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,586 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,588 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,590 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,592 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,594 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,596 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,598 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,600 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,602 copying build/lib/lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fil 2026-05-11T15:53:11,606 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,607 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,609 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,611 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_ja_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,613 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,615 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,617 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,619 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,621 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,623 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,625 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,627 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,629 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,632 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,634 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,636 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,638 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,640 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,642 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,644 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,646 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,648 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,650 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,652 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,654 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,656 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,658 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,659 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,661 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,663 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,665 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,667 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,669 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,671 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,673 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,675 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,677 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,679 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,681 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,683 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,685 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,687 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,689 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,691 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,693 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,695 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,697 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,698 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,700 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,702 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,704 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,705 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,707 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,709 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,711 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,712 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,714 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,716 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,718 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,720 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,721 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,723 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,725 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,727 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,729 copying build/lib/lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ja 2026-05-11T15:53:11,732 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,733 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,735 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,737 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,739 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,741 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,742 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,744 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,746 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,747 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,749 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,751 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_hi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,753 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,754 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,756 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,758 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,759 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,761 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,763 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,765 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,766 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,768 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,770 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,771 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,773 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,775 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,776 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,778 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,780 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,781 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,783 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,785 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,786 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,788 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,790 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,792 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,793 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,795 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,797 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,799 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,800 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,802 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,804 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,806 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,807 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,809 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,811 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,813 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,815 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,816 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,818 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,819 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,821 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,823 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,824 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,826 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,828 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,830 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,831 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,833 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,835 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,837 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,838 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,840 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,842 copying build/lib/lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/hi 2026-05-11T15:53:11,845 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,846 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,849 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,850 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,852 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,854 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,855 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,857 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,858 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,860 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,862 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,863 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,865 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,867 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,869 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,870 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,872 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,874 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,875 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,877 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,879 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,880 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,882 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_ms_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,884 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,886 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,887 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,889 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,891 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,892 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,894 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,896 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,897 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,899 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,901 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,902 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,904 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,906 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,908 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,909 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,911 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,913 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,914 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,916 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,918 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,919 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,921 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,923 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,925 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,926 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,928 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,930 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,931 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,933 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,934 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,936 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,938 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,939 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,941 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,942 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,944 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,946 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,948 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,949 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,951 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,953 copying build/lib/lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ms 2026-05-11T15:53:11,956 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,957 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,959 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,961 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,963 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,965 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,966 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,968 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,970 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,971 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,973 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,975 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,976 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,978 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,980 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,981 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,983 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,985 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,987 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,989 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,991 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,992 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,994 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,996 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:11,998 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,000 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,002 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,004 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,006 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,007 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,009 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,010 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,012 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,014 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,016 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,017 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,019 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,021 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,023 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,024 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,026 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,028 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,030 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,032 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,034 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,035 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,037 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,039 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,041 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,043 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,044 copying build/lib/lm_eval/tasks/global_mmlu/full/it/_it_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,046 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,047 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,049 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,051 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,052 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,054 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,056 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,058 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,060 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,061 copying build/lib/lm_eval/tasks/global_mmlu/full/it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,063 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,065 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,067 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,068 copying build/lib/lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/it 2026-05-11T15:53:12,071 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,072 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,075 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,076 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,078 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,080 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,082 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,083 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,085 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,087 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,088 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,090 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,092 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,094 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,095 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,097 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,099 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,101 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,102 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,104 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,106 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,108 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,109 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,111 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,113 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,115 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_sr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,117 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,118 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,120 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,122 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,123 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,125 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,126 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,128 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,130 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,132 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,133 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,135 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,136 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,138 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,140 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,142 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,144 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,145 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,147 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,149 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,151 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,152 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,154 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,156 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,157 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,159 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,161 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,162 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,164 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,166 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,167 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,169 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,171 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,173 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,175 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,176 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,178 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,180 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,181 copying build/lib/lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sr 2026-05-11T15:53:12,184 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,186 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,187 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,189 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,191 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,193 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,194 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,196 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,197 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,199 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,201 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,202 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,204 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,206 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,207 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,209 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,210 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,212 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,214 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,216 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,218 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,219 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_mg_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,221 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,223 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,224 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,226 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,228 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,230 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,231 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,233 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,235 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,236 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,238 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,240 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,241 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,243 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,244 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,246 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,248 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,250 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,251 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,253 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,255 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,256 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,258 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,260 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,262 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,263 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,265 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,267 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,269 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,271 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,273 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,274 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,276 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,278 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,280 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,282 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,283 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,285 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,287 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,289 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,291 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,293 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,295 copying build/lib/lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/mg 2026-05-11T15:53:12,298 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,300 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,302 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,304 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,306 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,308 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,310 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,312 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,313 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,315 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,317 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,319 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,321 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,323 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,325 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,327 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,328 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,330 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,332 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,334 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,336 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,338 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,340 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,342 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,344 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,346 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,348 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,350 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,352 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,353 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,355 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,357 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,359 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,361 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,363 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,364 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,366 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,368 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,370 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,372 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,374 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,376 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,378 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,380 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,382 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,383 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,386 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,387 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,390 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,391 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,393 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,395 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,397 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,399 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,401 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,402 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,404 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,406 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/_sv_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,408 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,410 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,411 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,413 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,415 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,417 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,419 copying build/lib/lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/sv 2026-05-11T15:53:12,423 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,424 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,426 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,427 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,429 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,431 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,433 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,435 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,437 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,439 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,441 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,443 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,444 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,446 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,448 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,450 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,452 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,454 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,455 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,457 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,460 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,461 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,463 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,465 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,467 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,469 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,471 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,474 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,476 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_fr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,479 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,482 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,484 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,486 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,488 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,490 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,492 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,493 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,495 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,497 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,499 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,501 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,503 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,505 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,506 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,508 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,510 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,512 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,515 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,517 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,519 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,522 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,524 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,526 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,528 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,530 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,533 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,535 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,537 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,540 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,542 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,545 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,547 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,550 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,552 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,554 copying build/lib/lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fr 2026-05-11T15:53:12,557 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,559 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,561 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,562 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,564 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,566 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,568 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,570 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,572 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,573 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,575 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,577 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,579 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,581 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,582 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,584 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,586 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,588 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,590 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,592 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,594 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,596 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,598 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,600 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,601 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,603 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,605 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,607 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,609 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,611 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,613 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,614 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,616 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,618 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,620 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,621 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_tr_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,623 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,625 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,627 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,628 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,630 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,632 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,634 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,636 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,638 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,640 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,641 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,643 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,645 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,647 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,649 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,651 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,653 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,654 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,656 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,658 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,660 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,661 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,663 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,665 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,667 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,669 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,671 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,672 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,674 copying build/lib/lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/tr 2026-05-11T15:53:12,678 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,679 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,681 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,683 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,684 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,686 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,688 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,690 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,692 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,694 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,695 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,697 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,699 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,701 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,703 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,705 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,706 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,708 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,710 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,712 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,714 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,716 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,718 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,720 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,722 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,724 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,726 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,727 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,729 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,731 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,733 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,734 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,736 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,738 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,740 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,741 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,743 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,745 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,747 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,749 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,750 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,752 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,754 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,756 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,758 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,760 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,762 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,764 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,765 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,767 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,769 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,771 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,773 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,774 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,776 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,778 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,780 copying build/lib/lm_eval/tasks/global_mmlu/full/el/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,782 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,784 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,785 copying build/lib/lm_eval/tasks/global_mmlu/full/el/_el_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,787 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,789 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,791 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,793 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,795 copying build/lib/lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/el 2026-05-11T15:53:12,799 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,800 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,802 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,805 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_he_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,806 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,808 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,810 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,812 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,813 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,815 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,817 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,818 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,820 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,822 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,824 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,825 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,827 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,829 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,830 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,832 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,834 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,836 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,837 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,839 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,841 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,843 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,844 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,846 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,848 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,850 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,851 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,853 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,854 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,856 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,858 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,859 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,861 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,863 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,865 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,866 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,868 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,870 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,872 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,873 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,875 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,877 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,879 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,880 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,882 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,884 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,886 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,887 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,889 copying build/lib/lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,890 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,892 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,894 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,895 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,897 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,899 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,900 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,902 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,904 copying build/lib/lm_eval/tasks/global_mmlu/full/he/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,906 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,907 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,909 copying build/lib/lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/he 2026-05-11T15:53:12,912 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,914 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,915 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,917 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,919 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,921 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,923 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,924 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,926 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,928 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,929 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,931 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,933 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,934 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,936 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,938 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,939 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,941 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,943 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,945 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,946 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,948 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,950 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,952 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,954 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,956 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,958 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,960 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_cs_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,962 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,963 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,965 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,967 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,968 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,970 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,972 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,974 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,975 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,977 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,979 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,981 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,983 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,984 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,986 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,988 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,990 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,992 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,994 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,995 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,997 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:12,999 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,001 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,003 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,005 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,006 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,008 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,010 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,012 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,013 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,015 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,017 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,019 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,021 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,023 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,025 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,026 copying build/lib/lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/cs 2026-05-11T15:53:13,030 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,031 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,034 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,036 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,038 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,040 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,042 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,044 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,045 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,047 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,049 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,051 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,053 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,055 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,057 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,058 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,060 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,062 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,064 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,065 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,067 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,069 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,071 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,072 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,074 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,076 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,078 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,080 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_pt_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,081 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,083 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,085 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,086 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,088 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,090 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,091 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,093 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,094 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,096 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,098 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,099 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,101 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,103 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,104 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,106 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,108 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,110 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,111 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,113 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,115 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,116 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,118 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,120 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,122 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,123 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,125 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,126 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,128 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,130 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,132 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,135 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,137 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,139 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,140 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,142 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,144 copying build/lib/lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/pt 2026-05-11T15:53:13,147 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,148 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,150 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,152 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,153 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,155 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_id_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,157 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,159 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,160 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,162 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,164 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,165 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,167 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,169 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,170 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,172 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,174 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,176 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,177 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,179 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,181 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,182 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,184 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,186 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,188 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,189 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,191 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,193 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,195 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,196 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,198 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,200 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,201 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,203 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,205 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,207 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,208 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,210 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,212 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,214 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,216 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,217 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,219 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,221 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,223 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,225 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,226 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,228 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,230 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,232 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,234 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,236 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,238 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,240 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,242 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,244 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,245 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,247 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,249 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,251 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,253 copying build/lib/lm_eval/tasks/global_mmlu/full/id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,255 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,257 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,259 copying build/lib/lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,261 copying build/lib/lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/id 2026-05-11T15:53:13,265 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,267 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,269 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,271 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,273 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_ar_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,275 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,277 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,280 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,282 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,284 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,286 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,288 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,290 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,291 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,293 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,295 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,297 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,300 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,302 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,304 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,306 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,308 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,310 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,312 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,314 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,316 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,318 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,320 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,322 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,324 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,326 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,329 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,331 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,333 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,335 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,337 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,339 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,341 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,343 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,345 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,347 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,349 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,351 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,353 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,355 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,357 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,359 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,361 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,363 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,365 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,367 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,370 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,372 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,374 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,376 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,378 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,380 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,382 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,384 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,386 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,388 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,391 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,393 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,395 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,397 copying build/lib/lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/ar 2026-05-11T15:53:13,401 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,402 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,404 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,407 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,409 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,411 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,413 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,415 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,417 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,419 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,421 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,424 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,426 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,428 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,430 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,432 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,434 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,436 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,438 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,440 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,443 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,445 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,447 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,449 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,451 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,453 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,455 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,458 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,460 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,462 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,464 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,466 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,468 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,470 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,472 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,474 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,476 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,479 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,481 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,483 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_fa_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,485 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,487 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,489 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,491 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,493 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,496 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,498 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,500 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,502 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,505 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,507 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,509 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,511 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,512 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,514 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,516 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,518 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,520 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,522 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,524 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,526 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,528 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,530 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,532 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,534 copying build/lib/lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/fa 2026-05-11T15:53:13,538 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,539 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,541 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,543 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,545 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,547 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,549 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,551 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,553 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,555 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,557 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,559 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,561 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,563 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,565 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,566 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,568 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,570 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,572 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,574 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,576 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,579 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,581 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,584 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,586 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,590 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,592 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,595 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,597 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,600 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,603 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,605 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,608 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,610 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,613 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,615 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,617 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,620 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,622 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,625 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,628 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,630 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,633 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,635 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,638 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,640 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,643 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,646 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,648 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,651 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,653 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,656 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,658 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,661 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_am_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,663 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,665 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,668 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,670 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,673 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,675 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,677 copying build/lib/lm_eval/tasks/global_mmlu/full/am/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,679 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,681 copying build/lib/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,683 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,685 copying build/lib/lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/global_mmlu/full/am 2026-05-11T15:53:13,688 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmlu 2026-05-11T15:53:13,690 copying build/lib/lm_eval/tasks/tmlu/subject.tsv -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu 2026-05-11T15:53:13,692 copying build/lib/lm_eval/tasks/tmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu 2026-05-11T15:53:13,695 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,696 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_nutritionist.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,698 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_lawyer_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,700 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_tour_guide.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,702 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,704 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,705 copying build/lib/lm_eval/tasks/tmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,707 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_basic_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,709 copying build/lib/lm_eval/tasks/tmlu/default/_tmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,711 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_driving_rule.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,712 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_accountant.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,714 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_tour_leader.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,716 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,717 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,720 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,722 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,724 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,726 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,728 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,730 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_clinical_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,733 copying build/lib/lm_eval/tasks/tmlu/default/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,736 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_civics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,738 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_taiwan_tourist_resources.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,740 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,743 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,745 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,747 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,750 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_clinical_psychologist.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,752 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,754 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_earth_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,756 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,758 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_CAP_earth_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,760 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,762 copying build/lib/lm_eval/tasks/tmlu/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,764 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_AST_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,766 copying build/lib/lm_eval/tasks/tmlu/default/tmlu_GSAT_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/tmlu/default 2026-05-11T15:53:13,768 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/drop 2026-05-11T15:53:13,769 copying build/lib/lm_eval/tasks/drop/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/drop 2026-05-11T15:53:13,771 copying build/lib/lm_eval/tasks/drop/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/drop 2026-05-11T15:53:13,773 copying build/lib/lm_eval/tasks/drop/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/drop 2026-05-11T15:53:13,775 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gsm_plus 2026-05-11T15:53:13,776 copying build/lib/lm_eval/tasks/gsm_plus/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm_plus 2026-05-11T15:53:13,778 copying build/lib/lm_eval/tasks/gsm_plus/gsm_plus_mini.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm_plus 2026-05-11T15:53:13,780 copying build/lib/lm_eval/tasks/gsm_plus/gsm_plus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gsm_plus 2026-05-11T15:53:13,784 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cmmlu 2026-05-11T15:53:13,785 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_food_culture.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,787 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,789 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,791 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_journalism.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,793 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,795 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,797 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_agronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,799 copying build/lib/lm_eval/tasks/cmmlu/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,800 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,802 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,804 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,806 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_legal_and_moral_basis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,808 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_food_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,810 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,812 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,814 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_literature.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,816 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,818 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,820 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,822 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,824 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_arts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,825 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,827 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,829 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,831 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,833 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,835 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,837 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_ancient_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,839 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_driving_rule.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,840 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_logical.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,842 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,844 copying build/lib/lm_eval/tasks/cmmlu/_cmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,846 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,849 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_information_and_technology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,851 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,853 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,854 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,856 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,858 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,860 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,862 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,864 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,866 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,868 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,870 copying build/lib/lm_eval/tasks/cmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,873 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_construction_project_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,875 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,877 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,879 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_civil_service_exam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,881 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,883 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,885 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,887 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,889 copying build/lib/lm_eval/tasks/cmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,891 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,893 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,896 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,899 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_chinese_teacher_qualification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,901 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,904 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,907 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,910 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,912 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,915 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,918 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,920 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,923 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,925 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,928 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_security_study.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,931 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_college_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,934 copying build/lib/lm_eval/tasks/cmmlu/cmmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cmmlu 2026-05-11T15:53:13,937 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/logiqa 2026-05-11T15:53:13,938 copying build/lib/lm_eval/tasks/logiqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa 2026-05-11T15:53:13,940 copying build/lib/lm_eval/tasks/logiqa/logiqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa 2026-05-11T15:53:13,942 copying build/lib/lm_eval/tasks/logiqa/utils_logiqa.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/logiqa 2026-05-11T15:53:13,945 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/race 2026-05-11T15:53:13,946 copying build/lib/lm_eval/tasks/race/race.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/race 2026-05-11T15:53:13,948 copying build/lib/lm_eval/tasks/race/preprocess_race.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/race 2026-05-11T15:53:13,950 copying build/lib/lm_eval/tasks/race/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/race 2026-05-11T15:53:13,953 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/spanish_bench 2026-05-11T15:53:13,955 copying build/lib/lm_eval/tasks/spanish_bench/spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:13,957 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:53:13,959 copying build/lib/lm_eval/tasks/spanish_bench/phrases_es/_phrases_es_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:53:13,961 copying build/lib/lm_eval/tasks/spanish_bench/phrases_es/phrases_va-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:53:13,963 copying build/lib/lm_eval/tasks/spanish_bench/phrases_es/phrases_es-va.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/phrases_es 2026-05-11T15:53:13,965 copying build/lib/lm_eval/tasks/spanish_bench/wnli_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:13,967 copying build/lib/lm_eval/tasks/spanish_bench/cocoteros_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:13,970 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,971 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_pt-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,973 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,975 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,977 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,979 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_it-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,981 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_eu-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,983 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_gl-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,985 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_en-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,987 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,989 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_ca-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,991 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,993 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_de-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,995 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,997 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:13,999 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/create_yamls_flores_es.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:14,002 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:14,004 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:14,006 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_es-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:14,007 copying build/lib/lm_eval/tasks/spanish_bench/flores_es/flores_fr-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench/flores_es 2026-05-11T15:53:14,009 copying build/lib/lm_eval/tasks/spanish_bench/mgsm_direct_es_spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,011 copying build/lib/lm_eval/tasks/spanish_bench/copa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,013 copying build/lib/lm_eval/tasks/spanish_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,016 copying build/lib/lm_eval/tasks/spanish_bench/escola.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,018 copying build/lib/lm_eval/tasks/spanish_bench/paws_es_spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,020 copying build/lib/lm_eval/tasks/spanish_bench/openbookqa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,022 copying build/lib/lm_eval/tasks/spanish_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,024 copying build/lib/lm_eval/tasks/spanish_bench/xlsum_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,026 copying build/lib/lm_eval/tasks/spanish_bench/xnli_es_spanish_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/spanish_bench 2026-05-11T15:53:14,029 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include 2026-05-11T15:53:14,031 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en 2026-05-11T15:53:14,033 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,034 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,036 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/_indonesian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,038 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,041 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,043 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,045 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/_include_base_44_indonesian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,047 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,049 copying build/lib/lm_eval/tasks/include/few_shot_en/Indonesian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Indonesian 2026-05-11T15:53:14,051 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,052 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,054 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/_include_base_44_bulgarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,056 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/_bulgarian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,058 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,060 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,062 copying build/lib/lm_eval/tasks/include/few_shot_en/Bulgarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bulgarian 2026-05-11T15:53:14,064 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,065 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/_bengali_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,067 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,069 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,071 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/_include_base_44_bengali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,073 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,075 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,077 copying build/lib/lm_eval/tasks/include/few_shot_en/Bengali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Bengali 2026-05-11T15:53:14,080 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,081 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/_include_base_44_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,083 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,085 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,087 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/_turkish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,089 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,091 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,093 copying build/lib/lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Turkish 2026-05-11T15:53:14,096 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,097 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,099 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,101 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/_include_base_44_belarusian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,102 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,104 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,106 copying build/lib/lm_eval/tasks/include/few_shot_en/Belarusian/_belarusian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Belarusian 2026-05-11T15:53:14,108 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,109 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,111 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,113 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,115 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/_greek_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,117 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,119 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,121 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/_include_base_44_greek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,123 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,125 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,127 copying build/lib/lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Greek 2026-05-11T15:53:14,130 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,131 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/_hungarian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,133 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/_include_base_44_hungarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,134 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,136 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,138 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,140 copying build/lib/lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hungarian 2026-05-11T15:53:14,142 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,143 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,145 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,147 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,149 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,151 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/_include_base_44_dutch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,153 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/_dutch_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,155 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,157 copying build/lib/lm_eval/tasks/include/few_shot_en/Dutch/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Dutch 2026-05-11T15:53:14,160 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:53:14,161 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:53:14,163 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/_korean_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:53:14,165 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/_include_base_44_korean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:53:14,167 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:53:14,169 copying build/lib/lm_eval/tasks/include/few_shot_en/Korean/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Korean 2026-05-11T15:53:14,171 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,172 copying build/lib/lm_eval/tasks/include/few_shot_en/German/include_base_44_german_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,174 copying build/lib/lm_eval/tasks/include/few_shot_en/German/_include_base_44_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,176 copying build/lib/lm_eval/tasks/include/few_shot_en/German/_german_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,178 copying build/lib/lm_eval/tasks/include/few_shot_en/German/include_base_44_german_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,180 copying build/lib/lm_eval/tasks/include/few_shot_en/German/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,182 copying build/lib/lm_eval/tasks/include/few_shot_en/German/include_base_44_german_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/German 2026-05-11T15:53:14,184 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,185 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/_polish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,187 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,189 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/_include_base_44_polish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,191 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,193 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,195 copying build/lib/lm_eval/tasks/include/few_shot_en/Polish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Polish 2026-05-11T15:53:14,197 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,198 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,200 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/_include_base_44_urdu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,202 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/_urdu_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,204 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,206 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,208 copying build/lib/lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Urdu 2026-05-11T15:53:14,210 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,212 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,214 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/_north macedonian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,216 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,218 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/_include_base_44_north macedonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,219 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,221 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,223 copying build/lib/lm_eval/tasks/include/few_shot_en/North Macedonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/North Macedonian 2026-05-11T15:53:14,226 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,227 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/_include_base_44_ukrainian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,229 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,231 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/_ukrainian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,233 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,235 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,237 copying build/lib/lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Ukrainian 2026-05-11T15:53:14,240 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,241 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,244 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,246 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,248 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,250 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/_chinese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,252 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,254 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,256 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,258 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,260 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,262 copying build/lib/lm_eval/tasks/include/few_shot_en/Chinese/_include_base_44_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Chinese 2026-05-11T15:53:14,264 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,266 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,268 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,270 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/_include_base_44_portuguese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,272 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/_portuguese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,273 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,275 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,277 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,279 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,281 copying build/lib/lm_eval/tasks/include/few_shot_en/Portuguese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Portuguese 2026-05-11T15:53:14,283 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:53:14,284 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/_georgian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:53:14,286 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/_include_base_44_georgian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:53:14,288 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:53:14,290 copying build/lib/lm_eval/tasks/include/few_shot_en/Georgian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Georgian 2026-05-11T15:53:14,293 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,294 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,296 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,298 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,300 copying build/lib/lm_eval/tasks/include/few_shot_en/French/_french_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,302 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,304 copying build/lib/lm_eval/tasks/include/few_shot_en/French/include_base_44_french_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,306 copying build/lib/lm_eval/tasks/include/few_shot_en/French/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,308 copying build/lib/lm_eval/tasks/include/few_shot_en/French/_include_base_44_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/French 2026-05-11T15:53:14,311 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,312 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/_spanish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,314 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,316 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/_include_base_44_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,317 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,319 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,321 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,323 copying build/lib/lm_eval/tasks/include/few_shot_en/Spanish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Spanish 2026-05-11T15:53:14,326 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,327 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,329 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/_serbian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,331 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,333 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/_include_base_44_serbian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,335 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,337 copying build/lib/lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Serbian 2026-05-11T15:53:14,340 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,341 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/_malay_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,343 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,345 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,347 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,349 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,350 copying build/lib/lm_eval/tasks/include/few_shot_en/Malay/_include_base_44_malay.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malay 2026-05-11T15:53:14,353 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,354 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/_arabic_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,356 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,358 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,360 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,361 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,363 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,365 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/_include_base_44_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,367 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,369 copying build/lib/lm_eval/tasks/include/few_shot_en/Arabic/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Arabic 2026-05-11T15:53:14,371 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,373 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,375 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/_vietnamese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,377 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,379 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/_include_base_44_vietnamese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,381 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,383 copying build/lib/lm_eval/tasks/include/few_shot_en/Vietnamese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Vietnamese 2026-05-11T15:53:14,386 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,387 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/_include_base_44_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,389 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,391 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,393 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,395 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,397 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,399 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,401 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,402 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/_russian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,404 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,406 copying build/lib/lm_eval/tasks/include/few_shot_en/Russian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Russian 2026-05-11T15:53:14,409 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,410 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,412 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/_include_base_44_japanese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,414 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,416 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/_japanese_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,418 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,420 copying build/lib/lm_eval/tasks/include/few_shot_en/Japanese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Japanese 2026-05-11T15:53:14,422 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,424 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,426 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,429 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,431 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,432 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,434 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,436 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/_azerbaijani_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,438 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,440 copying build/lib/lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Azerbaijani 2026-05-11T15:53:14,442 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,443 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,445 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,447 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,449 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,451 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/_include_base_44_persian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,453 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,455 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/_persian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,457 copying build/lib/lm_eval/tasks/include/few_shot_en/Persian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Persian 2026-05-11T15:53:14,460 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,461 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,463 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,465 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/_albanian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,467 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/_include_base_44_albanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,469 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,472 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,474 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,476 copying build/lib/lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Albanian 2026-05-11T15:53:14,478 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,480 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/_croatian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,481 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,483 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,485 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,487 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/_include_base_44_croatian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,489 copying build/lib/lm_eval/tasks/include/few_shot_en/Croatian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Croatian 2026-05-11T15:53:14,492 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:53:14,493 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:53:14,495 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/_nepali_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:53:14,496 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:53:14,498 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:53:14,500 copying build/lib/lm_eval/tasks/include/few_shot_en/Nepali/_include_base_44_nepali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Nepali 2026-05-11T15:53:14,503 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,504 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,506 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,508 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,510 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/_italian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,511 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/_include_base_44_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,513 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,515 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,517 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,518 copying build/lib/lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Italian 2026-05-11T15:53:14,521 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:53:14,522 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:53:14,524 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:53:14,526 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/_include_base_44_tamil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:53:14,527 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/_tamil_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:53:14,529 copying build/lib/lm_eval/tasks/include/few_shot_en/Tamil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tamil 2026-05-11T15:53:14,531 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:53:14,533 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/include_base_44_basque_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:53:14,535 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/_basque_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:53:14,537 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/_include_base_44_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:53:14,538 copying build/lib/lm_eval/tasks/include/few_shot_en/Basque/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Basque 2026-05-11T15:53:14,541 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:53:14,542 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:53:14,544 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/_include_base_44_hebrew.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:53:14,546 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/_hebrew_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:53:14,548 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:53:14,549 copying build/lib/lm_eval/tasks/include/few_shot_en/Hebrew/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hebrew 2026-05-11T15:53:14,552 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,553 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,555 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,556 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/_estonian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,558 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,560 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,563 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,564 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,566 copying build/lib/lm_eval/tasks/include/few_shot_en/Estonian/_include_base_44_estonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Estonian 2026-05-11T15:53:14,569 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,570 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,572 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,574 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,576 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/_include_base_44_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,578 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,580 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/_telugu_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,581 copying build/lib/lm_eval/tasks/include/few_shot_en/Telugu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Telugu 2026-05-11T15:53:14,584 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,585 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,587 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/_malayalam_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,589 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/_include_base_44_malayalam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,591 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,592 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,594 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,596 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,598 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,600 copying build/lib/lm_eval/tasks/include/few_shot_en/Malayalam/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Malayalam 2026-05-11T15:53:14,602 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:53:14,603 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/_include_base_44_kazakh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:53:14,605 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:53:14,607 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:53:14,609 copying build/lib/lm_eval/tasks/include/few_shot_en/Kazakh/_kazakh_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Kazakh 2026-05-11T15:53:14,611 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,612 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,614 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,616 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,618 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/_include_base_44_uzbek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,620 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/_uzbek_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,622 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,624 copying build/lib/lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Uzbek 2026-05-11T15:53:14,626 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,627 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,629 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,631 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/_armenian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,633 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,634 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/_include_base_44_armenian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,636 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,638 copying build/lib/lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Armenian 2026-05-11T15:53:14,640 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,641 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,643 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,645 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,647 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,649 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,651 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/_finnish_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,652 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/_include_base_44_finnish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,654 copying build/lib/lm_eval/tasks/include/few_shot_en/Finnish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Finnish 2026-05-11T15:53:14,657 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:53:14,658 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/_include_base_44_tagalog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:53:14,660 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:53:14,662 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/_tagalog_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:53:14,664 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:53:14,666 copying build/lib/lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Tagalog 2026-05-11T15:53:14,669 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,670 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,672 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,674 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,676 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/_include_base_44_lithuanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,678 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,679 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,681 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/_lithuanian_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,683 copying build/lib/lm_eval/tasks/include/few_shot_en/Lithuanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Lithuanian 2026-05-11T15:53:14,686 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,687 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/_hindi_few_shot_en_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,689 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,690 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,692 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,694 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,696 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,698 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,700 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,702 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,704 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/_include_base_44_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,705 copying build/lib/lm_eval/tasks/include/few_shot_en/Hindi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_en/Hindi 2026-05-11T15:53:14,709 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og 2026-05-11T15:53:14,711 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,712 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,714 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,716 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,718 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,719 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/_include_base_44_indonesian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,721 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/_indonesian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,723 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,725 copying build/lib/lm_eval/tasks/include/few_shot_og/Indonesian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Indonesian 2026-05-11T15:53:14,727 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,728 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,730 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/_include_base_44_bulgarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,732 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,734 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/_bulgarian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,735 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,737 copying build/lib/lm_eval/tasks/include/few_shot_og/Bulgarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bulgarian 2026-05-11T15:53:14,740 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,741 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/_bengali_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,743 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,745 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,747 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/_include_base_44_bengali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,749 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,750 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,752 copying build/lib/lm_eval/tasks/include/few_shot_og/Bengali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Bengali 2026-05-11T15:53:14,754 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,755 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/_include_base_44_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,757 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,759 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,761 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,763 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/_turkish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,765 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,767 copying build/lib/lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Turkish 2026-05-11T15:53:14,769 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,770 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/_belarusian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,772 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,774 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,776 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/_include_base_44_belarusian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,778 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,780 copying build/lib/lm_eval/tasks/include/few_shot_og/Belarusian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Belarusian 2026-05-11T15:53:14,783 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,784 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,786 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,787 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/_greek_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,789 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,791 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,793 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,794 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/_include_base_44_greek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,796 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,798 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,800 copying build/lib/lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Greek 2026-05-11T15:53:14,803 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,804 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/_hungarian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,806 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/_include_base_44_hungarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,807 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,809 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,811 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,813 copying build/lib/lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hungarian 2026-05-11T15:53:14,815 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,816 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,818 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,820 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,822 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,824 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/_include_base_44_dutch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,826 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/_dutch_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,827 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,829 copying build/lib/lm_eval/tasks/include/few_shot_og/Dutch/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Dutch 2026-05-11T15:53:14,831 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:53:14,832 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:53:14,835 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/_include_base_44_korean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:53:14,836 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:53:14,838 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/_korean_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:53:14,840 copying build/lib/lm_eval/tasks/include/few_shot_og/Korean/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Korean 2026-05-11T15:53:14,842 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,844 copying build/lib/lm_eval/tasks/include/few_shot_og/German/include_base_44_german_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,845 copying build/lib/lm_eval/tasks/include/few_shot_og/German/_include_base_44_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,847 copying build/lib/lm_eval/tasks/include/few_shot_og/German/_german_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,849 copying build/lib/lm_eval/tasks/include/few_shot_og/German/include_base_44_german_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,851 copying build/lib/lm_eval/tasks/include/few_shot_og/German/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,853 copying build/lib/lm_eval/tasks/include/few_shot_og/German/include_base_44_german_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/German 2026-05-11T15:53:14,855 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,856 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,858 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/_include_base_44_polish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,860 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,862 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,864 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/_polish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,866 copying build/lib/lm_eval/tasks/include/few_shot_og/Polish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Polish 2026-05-11T15:53:14,868 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,869 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,871 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/_urdu_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,872 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/_include_base_44_urdu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,874 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,876 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,877 copying build/lib/lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Urdu 2026-05-11T15:53:14,880 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,881 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,883 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,885 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/_include_base_44_north macedonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,886 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,888 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,890 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,892 copying build/lib/lm_eval/tasks/include/few_shot_og/North Macedonian/_north macedonian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/North Macedonian 2026-05-11T15:53:14,894 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,895 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/_ukrainian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,897 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/_include_base_44_ukrainian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,899 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,901 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,903 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,904 copying build/lib/lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Ukrainian 2026-05-11T15:53:14,907 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,908 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,910 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,911 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,913 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,915 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,917 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,918 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,920 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/_chinese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,922 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,924 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,926 copying build/lib/lm_eval/tasks/include/few_shot_og/Chinese/_include_base_44_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Chinese 2026-05-11T15:53:14,928 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,929 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/_portuguese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,931 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,934 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,935 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/_include_base_44_portuguese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,937 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,939 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,941 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,943 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,945 copying build/lib/lm_eval/tasks/include/few_shot_og/Portuguese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Portuguese 2026-05-11T15:53:14,947 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:53:14,948 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/_include_base_44_georgian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:53:14,950 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:53:14,952 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:53:14,953 copying build/lib/lm_eval/tasks/include/few_shot_og/Georgian/_georgian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Georgian 2026-05-11T15:53:14,956 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,957 copying build/lib/lm_eval/tasks/include/few_shot_og/French/_french_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,959 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,961 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,962 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,964 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,966 copying build/lib/lm_eval/tasks/include/few_shot_og/French/include_base_44_french_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,968 copying build/lib/lm_eval/tasks/include/few_shot_og/French/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,970 copying build/lib/lm_eval/tasks/include/few_shot_og/French/_include_base_44_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/French 2026-05-11T15:53:14,972 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,973 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/_spanish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,975 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,977 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/_include_base_44_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,979 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,980 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,982 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,984 copying build/lib/lm_eval/tasks/include/few_shot_og/Spanish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Spanish 2026-05-11T15:53:14,986 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,987 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,989 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,990 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/_serbian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,992 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/_include_base_44_serbian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,994 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,996 copying build/lib/lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Serbian 2026-05-11T15:53:14,998 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:14,999 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/_malay_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:15,001 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:15,003 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:15,005 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:15,006 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:15,008 copying build/lib/lm_eval/tasks/include/few_shot_og/Malay/_include_base_44_malay.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malay 2026-05-11T15:53:15,011 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,012 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,014 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,015 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,017 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,019 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,021 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/_arabic_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,023 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/_include_base_44_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,024 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,026 copying build/lib/lm_eval/tasks/include/few_shot_og/Arabic/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Arabic 2026-05-11T15:53:15,028 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,029 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,031 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/_vietnamese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,033 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,035 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/_include_base_44_vietnamese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,037 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,038 copying build/lib/lm_eval/tasks/include/few_shot_og/Vietnamese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Vietnamese 2026-05-11T15:53:15,041 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,042 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/_include_base_44_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,044 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,046 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,047 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,049 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,051 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,053 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,055 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,057 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/_russian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,058 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,060 copying build/lib/lm_eval/tasks/include/few_shot_og/Russian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Russian 2026-05-11T15:53:15,062 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,063 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/_japanese_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,065 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,067 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/_include_base_44_japanese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,069 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,070 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,072 copying build/lib/lm_eval/tasks/include/few_shot_og/Japanese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Japanese 2026-05-11T15:53:15,074 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,075 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,077 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,079 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,081 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,083 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/_azerbaijani_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,085 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,086 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,088 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,090 copying build/lib/lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Azerbaijani 2026-05-11T15:53:15,092 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,093 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,095 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,097 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,099 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,101 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/_include_base_44_persian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,102 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/_persian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,104 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,106 copying build/lib/lm_eval/tasks/include/few_shot_og/Persian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Persian 2026-05-11T15:53:15,109 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,110 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,112 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,113 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/_albanian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,115 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/_include_base_44_albanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,117 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,120 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,121 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,123 copying build/lib/lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Albanian 2026-05-11T15:53:15,126 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,127 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/_croatian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,129 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,131 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,133 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,135 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/_include_base_44_croatian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,136 copying build/lib/lm_eval/tasks/include/few_shot_og/Croatian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Croatian 2026-05-11T15:53:15,139 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:53:15,140 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:53:15,142 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/_nepali_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:53:15,144 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:53:15,145 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:53:15,147 copying build/lib/lm_eval/tasks/include/few_shot_og/Nepali/_include_base_44_nepali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Nepali 2026-05-11T15:53:15,150 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,151 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,153 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,154 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,156 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/_italian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,158 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/_include_base_44_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,160 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,162 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,164 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,165 copying build/lib/lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Italian 2026-05-11T15:53:15,168 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:53:15,169 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:53:15,171 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:53:15,173 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/_include_base_44_tamil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:53:15,175 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/_tamil_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:53:15,176 copying build/lib/lm_eval/tasks/include/few_shot_og/Tamil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tamil 2026-05-11T15:53:15,179 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:53:15,180 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/include_base_44_basque_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:53:15,181 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/_basque_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:53:15,183 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/_include_base_44_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:53:15,185 copying build/lib/lm_eval/tasks/include/few_shot_og/Basque/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Basque 2026-05-11T15:53:15,187 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:53:15,188 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:53:15,190 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/_include_base_44_hebrew.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:53:15,192 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:53:15,194 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/_hebrew_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:53:15,195 copying build/lib/lm_eval/tasks/include/few_shot_og/Hebrew/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hebrew 2026-05-11T15:53:15,198 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,199 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/_estonian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,201 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,202 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,204 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,206 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,208 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,209 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,211 copying build/lib/lm_eval/tasks/include/few_shot_og/Estonian/_include_base_44_estonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Estonian 2026-05-11T15:53:15,214 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,215 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,217 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,218 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,220 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/_telugu_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,222 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/_include_base_44_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,224 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,225 copying build/lib/lm_eval/tasks/include/few_shot_og/Telugu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Telugu 2026-05-11T15:53:15,228 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,229 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,231 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/_malayalam_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,233 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/_include_base_44_malayalam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,235 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,236 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,238 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,240 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,242 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,244 copying build/lib/lm_eval/tasks/include/few_shot_og/Malayalam/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Malayalam 2026-05-11T15:53:15,246 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:53:15,247 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/_include_base_44_kazakh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:53:15,249 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/_kazakh_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:53:15,251 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:53:15,253 copying build/lib/lm_eval/tasks/include/few_shot_og/Kazakh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Kazakh 2026-05-11T15:53:15,255 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,256 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,258 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,260 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/_uzbek_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,262 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,264 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/_include_base_44_uzbek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,265 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,267 copying build/lib/lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Uzbek 2026-05-11T15:53:15,270 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,271 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,273 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,275 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,277 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/_include_base_44_armenian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,278 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,280 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/_armenian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,282 copying build/lib/lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Armenian 2026-05-11T15:53:15,284 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,285 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,287 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,289 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,291 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,293 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,295 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/_finnish_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,297 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/_include_base_44_finnish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,298 copying build/lib/lm_eval/tasks/include/few_shot_og/Finnish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Finnish 2026-05-11T15:53:15,301 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:53:15,302 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/_include_base_44_tagalog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:53:15,304 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:53:15,306 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/_tagalog_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:53:15,308 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:53:15,309 copying build/lib/lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Tagalog 2026-05-11T15:53:15,312 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,313 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,315 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,316 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,318 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/_include_base_44_lithuanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,320 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,321 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/_lithuanian_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,323 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,325 copying build/lib/lm_eval/tasks/include/few_shot_og/Lithuanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Lithuanian 2026-05-11T15:53:15,327 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,329 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,331 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,333 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,334 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/_hindi_few_shot_og_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,336 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,338 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,341 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,343 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,344 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,346 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/_include_base_44_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,348 copying build/lib/lm_eval/tasks/include/few_shot_og/Hindi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/few_shot_og/Hindi 2026-05-11T15:53:15,350 copying build/lib/lm_eval/tasks/include/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include 2026-05-11T15:53:15,359 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default 2026-05-11T15:53:15,361 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,362 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,364 copying build/lib/lm_eval/tasks/include/default/Indonesian/_indonesian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,366 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,368 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,370 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,371 copying build/lib/lm_eval/tasks/include/default/Indonesian/_include_base_44_indonesian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,373 copying build/lib/lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,375 copying build/lib/lm_eval/tasks/include/default/Indonesian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Indonesian 2026-05-11T15:53:15,377 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,379 copying build/lib/lm_eval/tasks/include/default/Bulgarian/_bulgarian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,380 copying build/lib/lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,382 copying build/lib/lm_eval/tasks/include/default/Bulgarian/_include_base_44_bulgarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,384 copying build/lib/lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,385 copying build/lib/lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,387 copying build/lib/lm_eval/tasks/include/default/Bulgarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bulgarian 2026-05-11T15:53:15,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,391 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,392 copying build/lib/lm_eval/tasks/include/default/Bengali/_bengali_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,394 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,396 copying build/lib/lm_eval/tasks/include/default/Bengali/_include_base_44_bengali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,398 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,400 copying build/lib/lm_eval/tasks/include/default/Bengali/include_base_44_bengali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,401 copying build/lib/lm_eval/tasks/include/default/Bengali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Bengali 2026-05-11T15:53:15,404 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,405 copying build/lib/lm_eval/tasks/include/default/Turkish/_include_base_44_turkish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,407 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,408 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,410 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,412 copying build/lib/lm_eval/tasks/include/default/Turkish/_turkish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,414 copying build/lib/lm_eval/tasks/include/default/Turkish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,416 copying build/lib/lm_eval/tasks/include/default/Turkish/include_base_44_turkish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Turkish 2026-05-11T15:53:15,418 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,419 copying build/lib/lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,421 copying build/lib/lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,423 copying build/lib/lm_eval/tasks/include/default/Belarusian/_include_base_44_belarusian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,424 copying build/lib/lm_eval/tasks/include/default/Belarusian/_belarusian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,426 copying build/lib/lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,428 copying build/lib/lm_eval/tasks/include/default/Belarusian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Belarusian 2026-05-11T15:53:15,430 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,431 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,433 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,435 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,436 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,438 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,440 copying build/lib/lm_eval/tasks/include/default/Greek/_include_base_44_greek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,442 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,443 copying build/lib/lm_eval/tasks/include/default/Greek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,445 copying build/lib/lm_eval/tasks/include/default/Greek/include_base_44_greek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,447 copying build/lib/lm_eval/tasks/include/default/Greek/_greek_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Greek 2026-05-11T15:53:15,450 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,451 copying build/lib/lm_eval/tasks/include/default/Hungarian/_hungarian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,452 copying build/lib/lm_eval/tasks/include/default/Hungarian/_include_base_44_hungarian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,454 copying build/lib/lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,456 copying build/lib/lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,458 copying build/lib/lm_eval/tasks/include/default/Hungarian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,460 copying build/lib/lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hungarian 2026-05-11T15:53:15,462 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,463 copying build/lib/lm_eval/tasks/include/default/Dutch/_dutch_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,465 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,466 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,468 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,470 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,472 copying build/lib/lm_eval/tasks/include/default/Dutch/_include_base_44_dutch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,474 copying build/lib/lm_eval/tasks/include/default/Dutch/include_base_44_dutch_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,475 copying build/lib/lm_eval/tasks/include/default/Dutch/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Dutch 2026-05-11T15:53:15,478 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Korean 2026-05-11T15:53:15,479 copying build/lib/lm_eval/tasks/include/default/Korean/include_base_44_korean_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T15:53:15,481 copying build/lib/lm_eval/tasks/include/default/Korean/_include_base_44_korean.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T15:53:15,482 copying build/lib/lm_eval/tasks/include/default/Korean/_korean_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T15:53:15,484 copying build/lib/lm_eval/tasks/include/default/Korean/include_base_44_korean_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T15:53:15,486 copying build/lib/lm_eval/tasks/include/default/Korean/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Korean 2026-05-11T15:53:15,488 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/German 2026-05-11T15:53:15,489 copying build/lib/lm_eval/tasks/include/default/German/include_base_44_german_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T15:53:15,492 copying build/lib/lm_eval/tasks/include/default/German/_german_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T15:53:15,493 copying build/lib/lm_eval/tasks/include/default/German/_include_base_44_german.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T15:53:15,495 copying build/lib/lm_eval/tasks/include/default/German/include_base_44_german_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T15:53:15,497 copying build/lib/lm_eval/tasks/include/default/German/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T15:53:15,499 copying build/lib/lm_eval/tasks/include/default/German/include_base_44_german_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/German 2026-05-11T15:53:15,501 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,502 copying build/lib/lm_eval/tasks/include/default/Polish/include_base_44_polish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,504 copying build/lib/lm_eval/tasks/include/default/Polish/_include_base_44_polish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,506 copying build/lib/lm_eval/tasks/include/default/Polish/_polish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,507 copying build/lib/lm_eval/tasks/include/default/Polish/include_base_44_polish_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,509 copying build/lib/lm_eval/tasks/include/default/Polish/include_base_44_polish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,511 copying build/lib/lm_eval/tasks/include/default/Polish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Polish 2026-05-11T15:53:15,513 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,514 copying build/lib/lm_eval/tasks/include/default/Urdu/include_base_44_urdu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,516 copying build/lib/lm_eval/tasks/include/default/Urdu/_include_base_44_urdu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,518 copying build/lib/lm_eval/tasks/include/default/Urdu/include_base_44_urdu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,519 copying build/lib/lm_eval/tasks/include/default/Urdu/_urdu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,521 copying build/lib/lm_eval/tasks/include/default/Urdu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,523 copying build/lib/lm_eval/tasks/include/default/Urdu/include_base_44_urdu_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Urdu 2026-05-11T15:53:15,525 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,526 copying build/lib/lm_eval/tasks/include/default/North Macedonian/_north macedonian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,528 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,530 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,532 copying build/lib/lm_eval/tasks/include/default/North Macedonian/_include_base_44_north macedonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,534 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,535 copying build/lib/lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,537 copying build/lib/lm_eval/tasks/include/default/North Macedonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/North Macedonian 2026-05-11T15:53:15,540 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,541 copying build/lib/lm_eval/tasks/include/default/Ukrainian/_include_base_44_ukrainian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,543 copying build/lib/lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,544 copying build/lib/lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,546 copying build/lib/lm_eval/tasks/include/default/Ukrainian/_ukrainian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,548 copying build/lib/lm_eval/tasks/include/default/Ukrainian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,549 copying build/lib/lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Ukrainian 2026-05-11T15:53:15,552 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,553 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,554 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,556 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,558 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,560 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,561 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,563 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,565 copying build/lib/lm_eval/tasks/include/default/Chinese/include_base_44_chinese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,567 copying build/lib/lm_eval/tasks/include/default/Chinese/_chinese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,568 copying build/lib/lm_eval/tasks/include/default/Chinese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,570 copying build/lib/lm_eval/tasks/include/default/Chinese/_include_base_44_chinese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Chinese 2026-05-11T15:53:15,573 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,574 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,576 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,578 copying build/lib/lm_eval/tasks/include/default/Portuguese/_include_base_44_portuguese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,580 copying build/lib/lm_eval/tasks/include/default/Portuguese/_portuguese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,582 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,583 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,585 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,587 copying build/lib/lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,589 copying build/lib/lm_eval/tasks/include/default/Portuguese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Portuguese 2026-05-11T15:53:15,591 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Georgian 2026-05-11T15:53:15,592 copying build/lib/lm_eval/tasks/include/default/Georgian/_include_base_44_georgian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T15:53:15,594 copying build/lib/lm_eval/tasks/include/default/Georgian/_georgian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T15:53:15,596 copying build/lib/lm_eval/tasks/include/default/Georgian/include_base_44_georgian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T15:53:15,598 copying build/lib/lm_eval/tasks/include/default/Georgian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Georgian 2026-05-11T15:53:15,600 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/French 2026-05-11T15:53:15,601 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,603 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,605 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,607 copying build/lib/lm_eval/tasks/include/default/French/_french_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,609 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,611 copying build/lib/lm_eval/tasks/include/default/French/include_base_44_french_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,613 copying build/lib/lm_eval/tasks/include/default/French/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,615 copying build/lib/lm_eval/tasks/include/default/French/_include_base_44_french.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/French 2026-05-11T15:53:15,617 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,618 copying build/lib/lm_eval/tasks/include/default/Spanish/_spanish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,620 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,622 copying build/lib/lm_eval/tasks/include/default/Spanish/_include_base_44_spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,624 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,626 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,628 copying build/lib/lm_eval/tasks/include/default/Spanish/include_base_44_spanish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,630 copying build/lib/lm_eval/tasks/include/default/Spanish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Spanish 2026-05-11T15:53:15,632 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,633 copying build/lib/lm_eval/tasks/include/default/Serbian/include_base_44_serbian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,635 copying build/lib/lm_eval/tasks/include/default/Serbian/include_base_44_serbian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,637 copying build/lib/lm_eval/tasks/include/default/Serbian/_include_base_44_serbian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,639 copying build/lib/lm_eval/tasks/include/default/Serbian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,640 copying build/lib/lm_eval/tasks/include/default/Serbian/_serbian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,642 copying build/lib/lm_eval/tasks/include/default/Serbian/include_base_44_serbian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Serbian 2026-05-11T15:53:15,644 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,645 copying build/lib/lm_eval/tasks/include/default/Malay/include_base_44_malay_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,647 copying build/lib/lm_eval/tasks/include/default/Malay/include_base_44_malay_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,649 copying build/lib/lm_eval/tasks/include/default/Malay/_malay_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,651 copying build/lib/lm_eval/tasks/include/default/Malay/include_base_44_malay_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,653 copying build/lib/lm_eval/tasks/include/default/Malay/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,655 copying build/lib/lm_eval/tasks/include/default/Malay/_include_base_44_malay.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malay 2026-05-11T15:53:15,658 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,659 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,661 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,663 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,665 copying build/lib/lm_eval/tasks/include/default/Arabic/_arabic_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,666 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,668 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,670 copying build/lib/lm_eval/tasks/include/default/Arabic/_include_base_44_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,672 copying build/lib/lm_eval/tasks/include/default/Arabic/include_base_44_arabic_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,673 copying build/lib/lm_eval/tasks/include/default/Arabic/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Arabic 2026-05-11T15:53:15,676 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,679 copying build/lib/lm_eval/tasks/include/default/Vietnamese/_vietnamese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,681 copying build/lib/lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,682 copying build/lib/lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,684 copying build/lib/lm_eval/tasks/include/default/Vietnamese/_include_base_44_vietnamese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,686 copying build/lib/lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,688 copying build/lib/lm_eval/tasks/include/default/Vietnamese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Vietnamese 2026-05-11T15:53:15,690 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,691 copying build/lib/lm_eval/tasks/include/default/Russian/_russian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,693 copying build/lib/lm_eval/tasks/include/default/Russian/_include_base_44_russian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,695 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,697 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,699 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,701 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,702 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,704 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,706 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,708 copying build/lib/lm_eval/tasks/include/default/Russian/include_base_44_russian_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,710 copying build/lib/lm_eval/tasks/include/default/Russian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Russian 2026-05-11T15:53:15,712 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,713 copying build/lib/lm_eval/tasks/include/default/Japanese/_japanese_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,715 copying build/lib/lm_eval/tasks/include/default/Japanese/include_base_44_japanese_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,717 copying build/lib/lm_eval/tasks/include/default/Japanese/_include_base_44_japanese.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,719 copying build/lib/lm_eval/tasks/include/default/Japanese/include_base_44_japanese_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,721 copying build/lib/lm_eval/tasks/include/default/Japanese/include_base_44_japanese_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,723 copying build/lib/lm_eval/tasks/include/default/Japanese/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Japanese 2026-05-11T15:53:15,725 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,726 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,728 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,730 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,732 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,734 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/_azerbaijani_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,735 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/_include_base_44_azerbaijani.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,737 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,739 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,741 copying build/lib/lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Azerbaijani 2026-05-11T15:53:15,743 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,744 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,746 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,748 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,750 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,751 copying build/lib/lm_eval/tasks/include/default/Persian/_include_base_44_persian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,753 copying build/lib/lm_eval/tasks/include/default/Persian/include_base_44_persian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,755 copying build/lib/lm_eval/tasks/include/default/Persian/_persian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,757 copying build/lib/lm_eval/tasks/include/default/Persian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Persian 2026-05-11T15:53:15,759 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,760 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,762 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,764 copying build/lib/lm_eval/tasks/include/default/Albanian/_include_base_44_albanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,766 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,768 copying build/lib/lm_eval/tasks/include/default/Albanian/_albanian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,769 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,771 copying build/lib/lm_eval/tasks/include/default/Albanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,773 copying build/lib/lm_eval/tasks/include/default/Albanian/include_base_44_albanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Albanian 2026-05-11T15:53:15,775 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,776 copying build/lib/lm_eval/tasks/include/default/Croatian/_croatian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,778 copying build/lib/lm_eval/tasks/include/default/Croatian/include_base_44_croatian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,780 copying build/lib/lm_eval/tasks/include/default/Croatian/include_base_44_croatian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,782 copying build/lib/lm_eval/tasks/include/default/Croatian/include_base_44_croatian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,784 copying build/lib/lm_eval/tasks/include/default/Croatian/_include_base_44_croatian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,786 copying build/lib/lm_eval/tasks/include/default/Croatian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Croatian 2026-05-11T15:53:15,788 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Nepali 2026-05-11T15:53:15,789 copying build/lib/lm_eval/tasks/include/default/Nepali/_nepali_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T15:53:15,791 copying build/lib/lm_eval/tasks/include/default/Nepali/include_base_44_nepali_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T15:53:15,793 copying build/lib/lm_eval/tasks/include/default/Nepali/include_base_44_nepali_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T15:53:15,795 copying build/lib/lm_eval/tasks/include/default/Nepali/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T15:53:15,798 copying build/lib/lm_eval/tasks/include/default/Nepali/_include_base_44_nepali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Nepali 2026-05-11T15:53:15,801 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,802 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,804 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,807 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,809 copying build/lib/lm_eval/tasks/include/default/Italian/_italian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,811 copying build/lib/lm_eval/tasks/include/default/Italian/_include_base_44_italian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,813 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,815 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,818 copying build/lib/lm_eval/tasks/include/default/Italian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,820 copying build/lib/lm_eval/tasks/include/default/Italian/include_base_44_italian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Italian 2026-05-11T15:53:15,823 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Tamil 2026-05-11T15:53:15,824 copying build/lib/lm_eval/tasks/include/default/Tamil/include_base_44_tamil_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T15:53:15,827 copying build/lib/lm_eval/tasks/include/default/Tamil/include_base_44_tamil_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T15:53:15,829 copying build/lib/lm_eval/tasks/include/default/Tamil/_include_base_44_tamil.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T15:53:15,831 copying build/lib/lm_eval/tasks/include/default/Tamil/_tamil_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T15:53:15,834 copying build/lib/lm_eval/tasks/include/default/Tamil/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tamil 2026-05-11T15:53:15,837 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Basque 2026-05-11T15:53:15,838 copying build/lib/lm_eval/tasks/include/default/Basque/include_base_44_basque_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T15:53:15,840 copying build/lib/lm_eval/tasks/include/default/Basque/_basque_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T15:53:15,842 copying build/lib/lm_eval/tasks/include/default/Basque/_include_base_44_basque.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T15:53:15,844 copying build/lib/lm_eval/tasks/include/default/Basque/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Basque 2026-05-11T15:53:15,847 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Hebrew 2026-05-11T15:53:15,849 copying build/lib/lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T15:53:15,851 copying build/lib/lm_eval/tasks/include/default/Hebrew/_include_base_44_hebrew.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T15:53:15,853 copying build/lib/lm_eval/tasks/include/default/Hebrew/_hebrew_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T15:53:15,856 copying build/lib/lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T15:53:15,858 copying build/lib/lm_eval/tasks/include/default/Hebrew/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hebrew 2026-05-11T15:53:15,861 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,862 copying build/lib/lm_eval/tasks/include/default/Estonian/_estonian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,865 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,867 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,869 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,871 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,873 copying build/lib/lm_eval/tasks/include/default/Estonian/include_base_44_estonian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,876 copying build/lib/lm_eval/tasks/include/default/Estonian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,878 copying build/lib/lm_eval/tasks/include/default/Estonian/_include_base_44_estonian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Estonian 2026-05-11T15:53:15,881 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,882 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,885 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,887 copying build/lib/lm_eval/tasks/include/default/Telugu/_telugu_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,890 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,892 copying build/lib/lm_eval/tasks/include/default/Telugu/_include_base_44_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,893 copying build/lib/lm_eval/tasks/include/default/Telugu/include_base_44_telugu_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,895 copying build/lib/lm_eval/tasks/include/default/Telugu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Telugu 2026-05-11T15:53:15,898 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,899 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_marine_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,901 copying build/lib/lm_eval/tasks/include/default/Malayalam/_include_base_44_malayalam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,903 copying build/lib/lm_eval/tasks/include/default/Malayalam/_malayalam_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,905 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,907 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,909 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,910 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,912 copying build/lib/lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,914 copying build/lib/lm_eval/tasks/include/default/Malayalam/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Malayalam 2026-05-11T15:53:15,917 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Kazakh 2026-05-11T15:53:15,918 copying build/lib/lm_eval/tasks/include/default/Kazakh/_kazakh_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T15:53:15,920 copying build/lib/lm_eval/tasks/include/default/Kazakh/_include_base_44_kazakh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T15:53:15,922 copying build/lib/lm_eval/tasks/include/default/Kazakh/include_base_44_kazakh_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T15:53:15,924 copying build/lib/lm_eval/tasks/include/default/Kazakh/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Kazakh 2026-05-11T15:53:15,926 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,927 copying build/lib/lm_eval/tasks/include/default/Uzbek/_uzbek_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,930 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,932 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_medical_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,934 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,936 copying build/lib/lm_eval/tasks/include/default/Uzbek/_include_base_44_uzbek.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,938 copying build/lib/lm_eval/tasks/include/default/Uzbek/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,939 copying build/lib/lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Uzbek 2026-05-11T15:53:15,942 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,943 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,945 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,947 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,949 copying build/lib/lm_eval/tasks/include/default/Armenian/_armenian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,951 copying build/lib/lm_eval/tasks/include/default/Armenian/_include_base_44_armenian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,953 copying build/lib/lm_eval/tasks/include/default/Armenian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,955 copying build/lib/lm_eval/tasks/include/default/Armenian/include_base_44_armenian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Armenian 2026-05-11T15:53:15,958 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,959 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,961 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,963 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,965 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,967 copying build/lib/lm_eval/tasks/include/default/Finnish/_finnish_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,969 copying build/lib/lm_eval/tasks/include/default/Finnish/include_base_44_finnish_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,971 copying build/lib/lm_eval/tasks/include/default/Finnish/_include_base_44_finnish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,973 copying build/lib/lm_eval/tasks/include/default/Finnish/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Finnish 2026-05-11T15:53:15,976 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Tagalog 2026-05-11T15:53:15,977 copying build/lib/lm_eval/tasks/include/default/Tagalog/_include_base_44_tagalog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T15:53:15,979 copying build/lib/lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T15:53:15,981 copying build/lib/lm_eval/tasks/include/default/Tagalog/_tagalog_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T15:53:15,983 copying build/lib/lm_eval/tasks/include/default/Tagalog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T15:53:15,985 copying build/lib/lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Tagalog 2026-05-11T15:53:15,987 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:15,989 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:15,991 copying build/lib/lm_eval/tasks/include/default/Lithuanian/_lithuanian_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:15,993 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:15,995 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:15,997 copying build/lib/lm_eval/tasks/include/default/Lithuanian/_include_base_44_lithuanian.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:15,999 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:16,001 copying build/lib/lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_business_commerce.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:16,003 copying build/lib/lm_eval/tasks/include/default/Lithuanian/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Lithuanian 2026-05-11T15:53:16,005 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,006 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,008 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_health_oriented_education.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,010 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_general_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,012 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_professional_certification.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,014 copying build/lib/lm_eval/tasks/include/default/Hindi/_hindi_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,016 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_social_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,018 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_driving_license.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,020 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_arts_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,022 copying build/lib/lm_eval/tasks/include/default/Hindi/include_base_44_hindi_applied_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,024 copying build/lib/lm_eval/tasks/include/default/Hindi/_include_base_44_hindi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,027 copying build/lib/lm_eval/tasks/include/default/Hindi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/include/default/Hindi 2026-05-11T15:53:16,029 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/slr_bench 2026-05-11T15:53:16,030 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,032 copying build/lib/lm_eval/tasks/slr_bench/lm_eval_slr_bench.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,034 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_all.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,036 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,038 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_basic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,040 copying build/lib/lm_eval/tasks/slr_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,041 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,043 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,045 copying build/lib/lm_eval/tasks/slr_bench/slr_bench_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/slr_bench 2026-05-11T15:53:16,047 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,048 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,050 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,052 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,054 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,056 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,058 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,060 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,062 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,064 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,065 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,067 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,069 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc_common -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,071 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,073 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,074 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,076 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,077 copying build/lib/lm_eval/tasks/truthfulqa-multi/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,079 copying build/lib/lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,081 copying build/lib/lm_eval/tasks/truthfulqa-multi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/truthfulqa-multi 2026-05-11T15:53:16,084 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/aime 2026-05-11T15:53:16,085 copying build/lib/lm_eval/tasks/aime/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T15:53:16,086 copying build/lib/lm_eval/tasks/aime/aime24.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T15:53:16,088 copying build/lib/lm_eval/tasks/aime/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T15:53:16,091 copying build/lib/lm_eval/tasks/aime/aime.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T15:53:16,093 copying build/lib/lm_eval/tasks/aime/aime25.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/aime 2026-05-11T15:53:16,095 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eus_proficiency 2026-05-11T15:53:16,096 copying build/lib/lm_eval/tasks/eus_proficiency/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_proficiency 2026-05-11T15:53:16,098 copying build/lib/lm_eval/tasks/eus_proficiency/eus_proficiency.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eus_proficiency 2026-05-11T15:53:16,101 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr 2026-05-11T15:53:16,103 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,104 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,106 copying build/lib/lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,108 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,110 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,112 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,114 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,116 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,117 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,119 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,121 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,123 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,124 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,126 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,128 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,130 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,131 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,133 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,135 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,137 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,138 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,140 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,142 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,144 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,145 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,147 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,149 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,151 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,152 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,154 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,156 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,158 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,159 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,161 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,163 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,164 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,166 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,168 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,170 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,172 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,173 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,175 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,176 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,178 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,180 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,182 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,183 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,185 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,187 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,189 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,191 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,192 copying build/lib/lm_eval/tasks/mmlusr/question_only/_question_only.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,194 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,196 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,198 copying build/lib/lm_eval/tasks/mmlusr/question_only/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,199 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,201 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,203 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,204 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,206 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,208 copying build/lib/lm_eval/tasks/mmlusr/question_only/question_only_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_only 2026-05-11T15:53:16,210 copying build/lib/lm_eval/tasks/mmlusr/config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr 2026-05-11T15:53:16,213 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,214 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,216 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,218 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,220 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,222 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,223 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,225 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,227 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,229 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,231 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,232 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,234 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,236 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,238 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,240 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,241 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,243 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,245 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,247 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,248 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,250 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,252 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,254 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,256 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,258 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,259 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,261 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,263 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,265 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,267 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,269 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,270 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,272 copying build/lib/lm_eval/tasks/mmlusr/answer_only/_answer_only.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,274 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,276 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,278 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,280 copying build/lib/lm_eval/tasks/mmlusr/answer_only/_mmlusr_a_yml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,282 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,283 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,285 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,287 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,289 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,290 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,292 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,294 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,296 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,297 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,299 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,301 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,303 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,305 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,306 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,308 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,310 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,312 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,314 copying build/lib/lm_eval/tasks/mmlusr/answer_only/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,316 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,317 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,320 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,321 copying build/lib/lm_eval/tasks/mmlusr/answer_only/answer_only_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/answer_only 2026-05-11T15:53:16,323 copying build/lib/lm_eval/tasks/mmlusr/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr 2026-05-11T15:53:16,326 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,327 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,329 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,331 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,332 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,334 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,336 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,337 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,339 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,340 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,342 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,344 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,346 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,348 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,350 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,351 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,353 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,355 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,357 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,359 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,361 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,363 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,364 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,366 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,368 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,370 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,371 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,373 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,375 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/_question_and_answer.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,377 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,378 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,380 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,382 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,384 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,385 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,387 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,389 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,391 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,393 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,395 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,396 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,398 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,400 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,401 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/_mmlusr_qna_yml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,403 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,405 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,407 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,409 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,410 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,412 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,414 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,416 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,418 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,420 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,422 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,424 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,425 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,427 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,429 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,431 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,433 copying build/lib/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlusr/question_and_answer 2026-05-11T15:53:16,435 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue 2026-05-11T15:53:16,437 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,438 copying build/lib/lm_eval/tasks/code_x_glue/code-text/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,440 copying build/lib/lm_eval/tasks/code_x_glue/code-text/go.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,442 copying build/lib/lm_eval/tasks/code_x_glue/code-text/python.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,443 copying build/lib/lm_eval/tasks/code_x_glue/code-text/php.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,445 copying build/lib/lm_eval/tasks/code_x_glue/code-text/java.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,447 copying build/lib/lm_eval/tasks/code_x_glue/code-text/javascript.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,449 copying build/lib/lm_eval/tasks/code_x_glue/code-text/ruby.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,451 copying build/lib/lm_eval/tasks/code_x_glue/code-text/_codexglue.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,452 copying build/lib/lm_eval/tasks/code_x_glue/code-text/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,454 copying build/lib/lm_eval/tasks/code_x_glue/code-text/bleu.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,456 copying build/lib/lm_eval/tasks/code_x_glue/code-text/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/code_x_glue/code-text 2026-05-11T15:53:16,459 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/haerae 2026-05-11T15:53:16,460 copying build/lib/lm_eval/tasks/haerae/haerae_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,462 copying build/lib/lm_eval/tasks/haerae/_default_haerae_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,463 copying build/lib/lm_eval/tasks/haerae/haerae_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,465 copying build/lib/lm_eval/tasks/haerae/haerae_lw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,467 copying build/lib/lm_eval/tasks/haerae/haerae_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,469 copying build/lib/lm_eval/tasks/haerae/_haerae.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,471 copying build/lib/lm_eval/tasks/haerae/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,473 copying build/lib/lm_eval/tasks/haerae/haerae_gk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/haerae 2026-05-11T15:53:16,475 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/squadv2 2026-05-11T15:53:16,476 copying build/lib/lm_eval/tasks/squadv2/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squadv2 2026-05-11T15:53:16,479 copying build/lib/lm_eval/tasks/squadv2/squadv2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squadv2 2026-05-11T15:53:16,481 copying build/lib/lm_eval/tasks/squadv2/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/squadv2 2026-05-11T15:53:16,483 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/galician_bench 2026-05-11T15:53:16,485 copying build/lib/lm_eval/tasks/galician_bench/truthfulqa_gl_mc1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,488 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,489 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_fr-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,491 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/create_yamls_flores_gl.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,493 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,495 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_eu-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,497 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,498 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,500 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,502 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_it-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,504 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,505 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_en-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,507 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,509 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,511 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_pt-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,512 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_es-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,514 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,516 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,518 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_de-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,519 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_gl-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,521 copying build/lib/lm_eval/tasks/galician_bench/flores_gl/flores_ca-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench/flores_gl 2026-05-11T15:53:16,523 copying build/lib/lm_eval/tasks/galician_bench/belebele_glg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,525 copying build/lib/lm_eval/tasks/galician_bench/mgsm_direct_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,527 copying build/lib/lm_eval/tasks/galician_bench/parafrases_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,529 copying build/lib/lm_eval/tasks/galician_bench/truthfulqa_gl_mc2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,530 copying build/lib/lm_eval/tasks/galician_bench/galician_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,532 copying build/lib/lm_eval/tasks/galician_bench/galcola.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,534 copying build/lib/lm_eval/tasks/galician_bench/xstorycloze_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,536 copying build/lib/lm_eval/tasks/galician_bench/summarization_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,537 copying build/lib/lm_eval/tasks/galician_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,540 copying build/lib/lm_eval/tasks/galician_bench/openbookqa_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,541 copying build/lib/lm_eval/tasks/galician_bench/xnli_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,543 copying build/lib/lm_eval/tasks/galician_bench/truthfulqa_gl_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,545 copying build/lib/lm_eval/tasks/galician_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,547 copying build/lib/lm_eval/tasks/galician_bench/paws_gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/galician_bench 2026-05-11T15:53:16,550 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/piqa 2026-05-11T15:53:16,551 copying build/lib/lm_eval/tasks/piqa/piqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/piqa 2026-05-11T15:53:16,553 copying build/lib/lm_eval/tasks/piqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/piqa 2026-05-11T15:53:16,555 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench 2026-05-11T15:53:16,556 copying build/lib/lm_eval/tasks/eq_bench/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench 2026-05-11T15:53:16,558 copying build/lib/lm_eval/tasks/eq_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench 2026-05-11T15:53:16,561 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:53:16,562 copying build/lib/lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:53:16,564 copying build/lib/lm_eval/tasks/eq_bench/multilingual/eqbench_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:53:16,566 copying build/lib/lm_eval/tasks/eq_bench/multilingual/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench/multilingual 2026-05-11T15:53:16,567 copying build/lib/lm_eval/tasks/eq_bench/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/eq_bench 2026-05-11T15:53:16,570 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/portuguese_bench 2026-05-11T15:53:16,571 copying build/lib/lm_eval/tasks/portuguese_bench/assin_paraphrase.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T15:53:16,573 copying build/lib/lm_eval/tasks/portuguese_bench/portuguese_bench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T15:53:16,576 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,577 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_de-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,579 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,581 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,583 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_en-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,584 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-ca.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,586 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/create_yamls_flores_pt.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,588 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,590 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,591 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,594 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,595 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/_flores_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,597 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_gl-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,599 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_eu-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,600 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_it-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,602 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_es-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,604 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_ca-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,605 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-gl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,607 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_fr-pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,609 copying build/lib/lm_eval/tasks/portuguese_bench/flores_pt/flores_pt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench/flores_pt 2026-05-11T15:53:16,611 copying build/lib/lm_eval/tasks/portuguese_bench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T15:53:16,613 copying build/lib/lm_eval/tasks/portuguese_bench/assin_entailment.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/portuguese_bench 2026-05-11T15:53:16,615 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/basqueglue 2026-05-11T15:53:16,616 copying build/lib/lm_eval/tasks/basqueglue/wic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,618 copying build/lib/lm_eval/tasks/basqueglue/bhtc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,620 copying build/lib/lm_eval/tasks/basqueglue/qnli.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,622 copying build/lib/lm_eval/tasks/basqueglue/vaxx.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,624 copying build/lib/lm_eval/tasks/basqueglue/coref.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,626 copying build/lib/lm_eval/tasks/basqueglue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,628 copying build/lib/lm_eval/tasks/basqueglue/bec.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,629 copying build/lib/lm_eval/tasks/basqueglue/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/basqueglue 2026-05-11T15:53:16,632 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,633 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_repetitive_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,634 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_winobias_antistereotype.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,636 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_into_the_unknown.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,638 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_memo_trap.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,640 copying build/lib/lm_eval/tasks/inverse_scaling/_some_results -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,641 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_quote_repetition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,643 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_sig_figs.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,645 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_hindsight_neglect.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,647 copying build/lib/lm_eval/tasks/inverse_scaling/_inverse_scaling_mc_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,649 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_modus_tollens.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,651 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_neqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,652 copying build/lib/lm_eval/tasks/inverse_scaling/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,654 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_pattern_matching_suppression.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,656 copying build/lib/lm_eval/tasks/inverse_scaling/inverse_scaling_redefine_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/inverse_scaling 2026-05-11T15:53:16,658 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl 2026-05-11T15:53:16,660 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/reasoning 2026-05-11T15:53:16,661 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_p_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T15:53:16,663 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_q_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T15:53:16,665 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_statement_consistency_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T15:53:16,667 copying build/lib/lm_eval/tasks/kbl/reasoning/kbl_causal_reasoning_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T15:53:16,669 copying build/lib/lm_eval/tasks/kbl/reasoning/_kbl_reasoning_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/reasoning 2026-05-11T15:53:16,671 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,672 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_statute_number_and_content_matching_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,674 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_query_statute_matching_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,676 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_offense_component_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,678 copying build/lib/lm_eval/tasks/kbl/knowledge/_kbl_knowledge_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,679 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_statute_hallucination_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,681 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,683 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_legal_concept_qa_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,684 copying build/lib/lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_reasoning_em.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/knowledge 2026-05-11T15:53:16,687 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam 2026-05-11T15:53:16,689 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,690 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,692 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2011.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,694 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,695 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,697 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2010.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,699 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,701 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,702 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,704 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,706 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,708 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,710 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,711 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,713 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,715 copying build/lib/lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/responsibility 2026-05-11T15:53:16,717 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,718 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2024.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,720 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,722 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,724 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,725 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,727 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,729 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,731 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,732 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,734 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,736 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,738 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,740 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,741 copying build/lib/lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/criminal 2026-05-11T15:53:16,744 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,745 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,747 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,749 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2025.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,750 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,752 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,754 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,756 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,757 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,759 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,761 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,763 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,765 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2024.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,766 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,768 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,770 copying build/lib/lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/civil 2026-05-11T15:53:16,772 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,773 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2022.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,775 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2015.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,777 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,779 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,781 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2017.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,782 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2021.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,784 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2019.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,786 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2020.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,788 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/_base_em_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,790 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2024.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,792 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2016.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,794 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2018.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,796 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2023.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,797 copying build/lib/lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2014.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl/bar_exam/public 2026-05-11T15:53:16,799 copying build/lib/lm_eval/tasks/kbl/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/kbl 2026-05-11T15:53:16,802 copying build/lib/lm_eval/tasks/manager.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T15:53:16,805 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/egyhellaswag 2026-05-11T15:53:16,806 copying build/lib/lm_eval/tasks/egyhellaswag/egyhellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egyhellaswag 2026-05-11T15:53:16,808 copying build/lib/lm_eval/tasks/egyhellaswag/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egyhellaswag 2026-05-11T15:53:16,810 copying build/lib/lm_eval/tasks/egyhellaswag/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/egyhellaswag 2026-05-11T15:53:16,812 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmt2016 2026-05-11T15:53:16,813 copying build/lib/lm_eval/tasks/wmt2016/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmt2016 2026-05-11T15:53:16,815 copying build/lib/lm_eval/tasks/wmt2016/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmt2016 2026-05-11T15:53:16,817 copying build/lib/lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmt2016 2026-05-11T15:53:16,819 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mathqa 2026-05-11T15:53:16,820 copying build/lib/lm_eval/tasks/mathqa/mathqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mathqa 2026-05-11T15:53:16,822 copying build/lib/lm_eval/tasks/mathqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mathqa 2026-05-11T15:53:16,824 copying build/lib/lm_eval/tasks/mathqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mathqa 2026-05-11T15:53:16,827 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/qa4mre 2026-05-11T15:53:16,828 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2012.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T15:53:16,830 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2011.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T15:53:16,832 copying build/lib/lm_eval/tasks/qa4mre/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T15:53:16,834 copying build/lib/lm_eval/tasks/qa4mre/qa4mre_2013.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T15:53:16,836 copying build/lib/lm_eval/tasks/qa4mre/preprocess_qa4mre.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/qa4mre 2026-05-11T15:53:16,839 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pisa 2026-05-11T15:53:16,841 copying build/lib/lm_eval/tasks/pisa/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,843 copying build/lib/lm_eval/tasks/pisa/pisa_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,845 copying build/lib/lm_eval/tasks/pisa/pisa_ch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,847 copying build/lib/lm_eval/tasks/pisa/pisa_es_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,849 copying build/lib/lm_eval/tasks/pisa/pisa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,851 copying build/lib/lm_eval/tasks/pisa/_pisa_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,853 copying build/lib/lm_eval/tasks/pisa/pisa_en_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,856 copying build/lib/lm_eval/tasks/pisa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,858 copying build/lib/lm_eval/tasks/pisa/pisa_fr_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,860 copying build/lib/lm_eval/tasks/pisa/_pisa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,862 copying build/lib/lm_eval/tasks/pisa/pisa_it_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,864 copying build/lib/lm_eval/tasks/pisa/pisa_de_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,867 copying build/lib/lm_eval/tasks/pisa/pisa_ch_llm_judged.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,869 copying build/lib/lm_eval/tasks/pisa/pisa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,871 copying build/lib/lm_eval/tasks/pisa/pisa_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,873 copying build/lib/lm_eval/tasks/pisa/pisa_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,875 copying build/lib/lm_eval/tasks/pisa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pisa 2026-05-11T15:53:16,879 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,880 copying build/lib/lm_eval/tasks/xstorycloze/default_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,882 copying build/lib/lm_eval/tasks/xstorycloze/default_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,884 copying build/lib/lm_eval/tasks/xstorycloze/default_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,887 copying build/lib/lm_eval/tasks/xstorycloze/_xstorycloze.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,889 copying build/lib/lm_eval/tasks/xstorycloze/default_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,891 copying build/lib/lm_eval/tasks/xstorycloze/default_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,893 copying build/lib/lm_eval/tasks/xstorycloze/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,895 copying build/lib/lm_eval/tasks/xstorycloze/default_my.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,897 copying build/lib/lm_eval/tasks/xstorycloze/default_eu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,899 copying build/lib/lm_eval/tasks/xstorycloze/default_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,901 copying build/lib/lm_eval/tasks/xstorycloze/default_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,903 copying build/lib/lm_eval/tasks/xstorycloze/default_te.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,905 copying build/lib/lm_eval/tasks/xstorycloze/default_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xstorycloze 2026-05-11T15:53:16,908 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/esbbq 2026-05-11T15:53:16,909 copying build/lib/lm_eval/tasks/esbbq/esbbq_lgbtqia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,911 copying build/lib/lm_eval/tasks/esbbq/esbbq_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,913 copying build/lib/lm_eval/tasks/esbbq/esbbq_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,915 copying build/lib/lm_eval/tasks/esbbq/esbbq_ses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,917 copying build/lib/lm_eval/tasks/esbbq/esbbq_spanish_region.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,919 copying build/lib/lm_eval/tasks/esbbq/esbbq_disability_status.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,922 copying build/lib/lm_eval/tasks/esbbq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,924 copying build/lib/lm_eval/tasks/esbbq/esbbq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,926 copying build/lib/lm_eval/tasks/esbbq/esbbq_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,928 copying build/lib/lm_eval/tasks/esbbq/_esbbq_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,930 copying build/lib/lm_eval/tasks/esbbq/esbbq_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,932 copying build/lib/lm_eval/tasks/esbbq/esbbq_race_ethnicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,934 copying build/lib/lm_eval/tasks/esbbq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,936 copying build/lib/lm_eval/tasks/esbbq/esbbq_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/esbbq 2026-05-11T15:53:16,939 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks 2026-05-11T15:53:16,941 copying build/lib/lm_eval/tasks/benchmarks/openllm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T15:53:16,943 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/multimedqa 2026-05-11T15:53:16,945 copying build/lib/lm_eval/tasks/benchmarks/multimedqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/multimedqa 2026-05-11T15:53:16,947 copying build/lib/lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/multimedqa 2026-05-11T15:53:16,949 copying build/lib/lm_eval/tasks/benchmarks/t0_eval.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T15:53:16,952 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/benchmarks/flan 2026-05-11T15:53:16,954 copying build/lib/lm_eval/tasks/benchmarks/flan/_held_in_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/flan 2026-05-11T15:53:16,956 copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_in.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/flan 2026-05-11T15:53:16,959 copying build/lib/lm_eval/tasks/benchmarks/flan/flan_held_out.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks/flan 2026-05-11T15:53:16,961 copying build/lib/lm_eval/tasks/benchmarks/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T15:53:16,963 copying build/lib/lm_eval/tasks/benchmarks/minerva_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T15:53:16,965 copying build/lib/lm_eval/tasks/benchmarks/pythia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/benchmarks 2026-05-11T15:53:16,969 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,970 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10proc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,972 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,974 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,976 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9cm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,979 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_atc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,981 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,983 copying build/lib/lm_eval/tasks/med_concepts_qa/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,985 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,987 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9proc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,989 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,991 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,994 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,996 copying build/lib/lm_eval/tasks/med_concepts_qa/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:16,998 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,000 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,002 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,004 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,006 copying build/lib/lm_eval/tasks/med_concepts_qa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,009 copying build/lib/lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10cm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,011 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,013 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_hard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,015 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,017 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_easy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,019 copying build/lib/lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_medium.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/med_concepts_qa 2026-05-11T15:53:17,022 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/groundcocoa 2026-05-11T15:53:17,023 copying build/lib/lm_eval/tasks/groundcocoa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/groundcocoa 2026-05-11T15:53:17,025 copying build/lib/lm_eval/tasks/groundcocoa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/groundcocoa 2026-05-11T15:53:17,027 copying build/lib/lm_eval/tasks/groundcocoa/groundcocoa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/groundcocoa 2026-05-11T15:53:17,032 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,033 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,035 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,038 copying build/lib/lm_eval/tasks/evalita_llm/_sum_template_fp-small_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,040 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,042 copying build/lib/lm_eval/tasks/evalita_llm/_ner_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,044 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,046 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,048 copying build/lib/lm_eval/tasks/evalita_llm/_hs_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,050 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,053 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,055 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ls_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,057 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,059 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,061 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_re_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,063 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,066 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,068 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,070 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,072 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,074 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,077 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,079 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,081 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,084 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,086 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,088 copying build/lib/lm_eval/tasks/evalita_llm/_wic_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,090 copying build/lib/lm_eval/tasks/evalita_llm/_ls_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,092 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,095 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,097 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,099 copying build/lib/lm_eval/tasks/evalita_llm/sum_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,101 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,103 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,105 copying build/lib/lm_eval/tasks/evalita_llm/_re_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,108 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,110 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,112 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,114 copying build/lib/lm_eval/tasks/evalita_llm/_faq_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,116 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,118 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,121 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,123 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,125 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,127 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,129 copying build/lib/lm_eval/tasks/evalita_llm/_sum_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,131 copying build/lib/lm_eval/tasks/evalita_llm/_sa_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,134 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,136 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,138 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,141 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ls_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,143 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,145 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,148 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_re_task.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,151 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,153 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,155 copying build/lib/lm_eval/tasks/evalita_llm/_sa_template_v2_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,157 copying build/lib/lm_eval/tasks/evalita_llm/_te_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,159 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,161 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,163 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,165 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,166 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,168 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,170 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,172 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_re_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,174 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,175 copying build/lib/lm_eval/tasks/evalita_llm/_sum_template_fp_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,177 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,179 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,181 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,183 copying build/lib/lm_eval/tasks/evalita_llm/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,185 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,187 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_p5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,189 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_mc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,191 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_wic_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,193 copying build/lib/lm_eval/tasks/evalita_llm/_at_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,195 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_faq_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,197 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ls_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,199 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,201 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,203 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_te_tasks.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,205 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,207 copying build/lib/lm_eval/tasks/evalita_llm/metrics.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,209 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,211 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,213 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_hs_p4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,215 copying build/lib/lm_eval/tasks/evalita_llm/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,218 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_sa_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,220 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,222 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_gen.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,223 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,225 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,227 copying build/lib/lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/evalita_llm 2026-05-11T15:53:17,230 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/medtext 2026-05-11T15:53:17,231 copying build/lib/lm_eval/tasks/medtext/medtext.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T15:53:17,233 copying build/lib/lm_eval/tasks/medtext/medtext_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T15:53:17,235 copying build/lib/lm_eval/tasks/medtext/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T15:53:17,237 copying build/lib/lm_eval/tasks/medtext/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T15:53:17,239 copying build/lib/lm_eval/tasks/medtext/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/medtext 2026-05-11T15:53:17,242 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/wmdp 2026-05-11T15:53:17,243 copying build/lib/lm_eval/tasks/wmdp/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T15:53:17,245 copying build/lib/lm_eval/tasks/wmdp/wmdp_chem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T15:53:17,247 copying build/lib/lm_eval/tasks/wmdp/wmdp_bio.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T15:53:17,249 copying build/lib/lm_eval/tasks/wmdp/wmdp_cyber.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T15:53:17,251 copying build/lib/lm_eval/tasks/wmdp/_wmdp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T15:53:17,253 copying build/lib/lm_eval/tasks/wmdp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/wmdp 2026-05-11T15:53:17,256 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xquad 2026-05-11T15:53:17,257 copying build/lib/lm_eval/tasks/xquad/xquad_ro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,259 copying build/lib/lm_eval/tasks/xquad/xquad_el.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,261 copying build/lib/lm_eval/tasks/xquad/xquad_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,263 copying build/lib/lm_eval/tasks/xquad/xquad_ar.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,265 copying build/lib/lm_eval/tasks/xquad/xquad_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,267 copying build/lib/lm_eval/tasks/xquad/xquad_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,269 copying build/lib/lm_eval/tasks/xquad/xquad_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,270 copying build/lib/lm_eval/tasks/xquad/xquad_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,272 copying build/lib/lm_eval/tasks/xquad/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,274 copying build/lib/lm_eval/tasks/xquad/xquad_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,276 copying build/lib/lm_eval/tasks/xquad/xquad_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,278 copying build/lib/lm_eval/tasks/xquad/xquad_hi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,280 copying build/lib/lm_eval/tasks/xquad/xquad_ru.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,281 copying build/lib/lm_eval/tasks/xquad/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,283 copying build/lib/lm_eval/tasks/xquad/xquad_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xquad 2026-05-11T15:53:17,286 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/paws-x 2026-05-11T15:53:17,287 copying build/lib/lm_eval/tasks/paws-x/paws_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,289 copying build/lib/lm_eval/tasks/paws-x/_pawsx.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,291 copying build/lib/lm_eval/tasks/paws-x/pawsx_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,293 copying build/lib/lm_eval/tasks/paws-x/paws_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,294 copying build/lib/lm_eval/tasks/paws-x/paws_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,296 copying build/lib/lm_eval/tasks/paws-x/paws_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,298 copying build/lib/lm_eval/tasks/paws-x/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,300 copying build/lib/lm_eval/tasks/paws-x/_generate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,302 copying build/lib/lm_eval/tasks/paws-x/paws_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,304 copying build/lib/lm_eval/tasks/paws-x/paws_ja.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,306 copying build/lib/lm_eval/tasks/paws-x/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,308 copying build/lib/lm_eval/tasks/paws-x/paws_ko.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/paws-x 2026-05-11T15:53:17,310 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/pile 2026-05-11T15:53:17,312 copying build/lib/lm_eval/tasks/pile/pile_github.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,314 copying build/lib/lm_eval/tasks/pile/pile_bookcorpus2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,316 copying build/lib/lm_eval/tasks/pile/pile_stackexchange.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,318 copying build/lib/lm_eval/tasks/pile/pile_hackernews.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,320 copying build/lib/lm_eval/tasks/pile/pile_opensubtitles.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,321 copying build/lib/lm_eval/tasks/pile/pile_uspto.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,323 copying build/lib/lm_eval/tasks/pile/pile_enron.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,325 copying build/lib/lm_eval/tasks/pile/pile_books3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,327 copying build/lib/lm_eval/tasks/pile/pile_pubmed-central.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,329 copying build/lib/lm_eval/tasks/pile/pile_wikipedia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,330 copying build/lib/lm_eval/tasks/pile/pile_openwebtext2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,332 copying build/lib/lm_eval/tasks/pile/pile_philpapers.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,334 copying build/lib/lm_eval/tasks/pile/pile_youtubesubtitles.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,336 copying build/lib/lm_eval/tasks/pile/pile_nih-exporter.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,337 copying build/lib/lm_eval/tasks/pile/pile_ubuntu-irc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,339 copying build/lib/lm_eval/tasks/pile/pile_dm-mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,341 copying build/lib/lm_eval/tasks/pile/pile_pile-cc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,343 copying build/lib/lm_eval/tasks/pile/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,345 copying build/lib/lm_eval/tasks/pile/pile_europarl.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,347 copying build/lib/lm_eval/tasks/pile/pile_arxiv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,349 copying build/lib/lm_eval/tasks/pile/pile_gutenberg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,351 copying build/lib/lm_eval/tasks/pile/pile_pubmed-abstracts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,353 copying build/lib/lm_eval/tasks/pile/pile_freelaw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/pile 2026-05-11T15:53:17,355 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/xcopa 2026-05-11T15:53:17,356 copying build/lib/lm_eval/tasks/xcopa/default_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,358 copying build/lib/lm_eval/tasks/xcopa/default_ta.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,360 copying build/lib/lm_eval/tasks/xcopa/default_qu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,362 copying build/lib/lm_eval/tasks/xcopa/default_vi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,364 copying build/lib/lm_eval/tasks/xcopa/default_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,366 copying build/lib/lm_eval/tasks/xcopa/_xcopa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,367 copying build/lib/lm_eval/tasks/xcopa/default_et.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,370 copying build/lib/lm_eval/tasks/xcopa/default_th.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,371 copying build/lib/lm_eval/tasks/xcopa/default_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,373 copying build/lib/lm_eval/tasks/xcopa/default_zh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,375 copying build/lib/lm_eval/tasks/xcopa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,377 copying build/lib/lm_eval/tasks/xcopa/default_ht.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,379 copying build/lib/lm_eval/tasks/xcopa/default_tr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,381 copying build/lib/lm_eval/tasks/xcopa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/xcopa 2026-05-11T15:53:17,384 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/turblimp 2026-05-11T15:53:17,385 copying build/lib/lm_eval/tasks/turblimp/determiners.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,387 copying build/lib/lm_eval/tasks/turblimp/anaphor_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,389 copying build/lib/lm_eval/tasks/turblimp/argument_structure_transitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,390 copying build/lib/lm_eval/tasks/turblimp/scrambling.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,392 copying build/lib/lm_eval/tasks/turblimp/_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,394 copying build/lib/lm_eval/tasks/turblimp/turblimp_group.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,396 copying build/lib/lm_eval/tasks/turblimp/argument_structure_ditransitive.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,398 copying build/lib/lm_eval/tasks/turblimp/irregular_forms.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,399 copying build/lib/lm_eval/tasks/turblimp/relative_clauses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,401 copying build/lib/lm_eval/tasks/turblimp/suspended_affixation.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,403 copying build/lib/lm_eval/tasks/turblimp/nominalization.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,405 copying build/lib/lm_eval/tasks/turblimp/npi_licensing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,407 copying build/lib/lm_eval/tasks/turblimp/island_effects.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,409 copying build/lib/lm_eval/tasks/turblimp/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,411 copying build/lib/lm_eval/tasks/turblimp/passives.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,412 copying build/lib/lm_eval/tasks/turblimp/binding.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,414 copying build/lib/lm_eval/tasks/turblimp/quantifiers.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,416 copying build/lib/lm_eval/tasks/turblimp/subject_agreement.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,418 copying build/lib/lm_eval/tasks/turblimp/ellipsis.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/turblimp 2026-05-11T15:53:17,420 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue 2026-05-11T15:53:17,422 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qqp 2026-05-11T15:53:17,423 copying build/lib/lm_eval/tasks/glue/qqp/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/qqp 2026-05-11T15:53:17,426 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/qnli 2026-05-11T15:53:17,427 copying build/lib/lm_eval/tasks/glue/qnli/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/qnli 2026-05-11T15:53:17,430 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/wnli 2026-05-11T15:53:17,431 copying build/lib/lm_eval/tasks/glue/wnli/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/wnli 2026-05-11T15:53:17,434 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/sst2 2026-05-11T15:53:17,435 copying build/lib/lm_eval/tasks/glue/sst2/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/sst2 2026-05-11T15:53:17,437 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/rte 2026-05-11T15:53:17,438 copying build/lib/lm_eval/tasks/glue/rte/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/rte 2026-05-11T15:53:17,441 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mrpc 2026-05-11T15:53:17,442 copying build/lib/lm_eval/tasks/glue/mrpc/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mrpc 2026-05-11T15:53:17,444 copying build/lib/lm_eval/tasks/glue/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue 2026-05-11T15:53:17,447 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/mnli 2026-05-11T15:53:17,448 copying build/lib/lm_eval/tasks/glue/mnli/mismatch.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mnli 2026-05-11T15:53:17,450 copying build/lib/lm_eval/tasks/glue/mnli/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mnli 2026-05-11T15:53:17,451 copying build/lib/lm_eval/tasks/glue/mnli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/mnli 2026-05-11T15:53:17,454 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/glue/cola 2026-05-11T15:53:17,455 copying build/lib/lm_eval/tasks/glue/cola/default.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/glue/cola 2026-05-11T15:53:17,457 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa 2026-05-11T15:53:17,459 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,460 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,462 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,464 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,466 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,468 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,470 copying build/lib/lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_n_shot 2026-05-11T15:53:17,472 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,473 copying build/lib/lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,475 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,477 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,479 copying build/lib/lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,481 copying build/lib/lm_eval/tasks/gpqa/n_shot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,483 copying build/lib/lm_eval/tasks/gpqa/n_shot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/n_shot 2026-05-11T15:53:17,485 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,486 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,489 copying build/lib/lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,491 copying build/lib/lm_eval/tasks/gpqa/generative/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,493 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,494 copying build/lib/lm_eval/tasks/gpqa/generative/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,496 copying build/lib/lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/generative 2026-05-11T15:53:17,498 copying build/lib/lm_eval/tasks/gpqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa 2026-05-11T15:53:17,500 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,501 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,503 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,505 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,507 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,509 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,510 copying build/lib/lm_eval/tasks/gpqa/cot_zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/cot_zeroshot 2026-05-11T15:53:17,513 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,514 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,516 copying build/lib/lm_eval/tasks/gpqa/zeroshot/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,518 copying build/lib/lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,519 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,521 copying build/lib/lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,523 copying build/lib/lm_eval/tasks/gpqa/zeroshot/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/gpqa/zeroshot 2026-05-11T15:53:17,525 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench 2026-05-11T15:53:17,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc 2026-05-11T15:53:17,529 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:53:17,530 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:53:17,532 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:53:17,534 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:53:17,536 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_2 2026-05-11T15:53:17,538 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:53:17,539 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:53:17,541 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:53:17,543 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:53:17,545 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_1 2026-05-11T15:53:17,547 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:53:17,548 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:53:17,550 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:53:17,552 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:53:17,553 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_5 2026-05-11T15:53:17,555 copying build/lib/lm_eval/tasks/afrobench/naijarc/naijarc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc 2026-05-11T15:53:17,557 copying build/lib/lm_eval/tasks/afrobench/naijarc/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc 2026-05-11T15:53:17,560 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:53:17,561 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:53:17,563 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:53:17,564 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:53:17,566 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_3 2026-05-11T15:53:17,569 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:53:17,570 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:53:17,572 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:53:17,574 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:53:17,575 copying build/lib/lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc/prompt_4 2026-05-11T15:53:17,577 copying build/lib/lm_eval/tasks/afrobench/naijarc/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/naijarc 2026-05-11T15:53:17,580 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib 2026-05-11T15:53:17,582 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,583 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,585 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,587 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,589 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,591 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,592 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,594 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,596 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,598 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,599 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,601 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,603 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,605 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,606 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,608 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,610 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,612 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,614 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,615 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,617 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,619 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,621 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,623 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,625 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,626 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,628 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,630 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,632 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,634 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,636 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,637 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,639 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,641 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,643 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,645 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,646 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,648 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,650 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,652 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,654 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,656 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,658 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,660 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,662 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,663 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,665 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,667 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,668 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,670 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,672 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,674 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,676 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,678 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,679 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,681 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,683 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,685 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,687 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_2/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_2 2026-05-11T15:53:17,690 copying build/lib/lm_eval/tasks/afrobench/sib/sib.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib 2026-05-11T15:53:17,693 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,694 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,696 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,698 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,700 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,701 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,703 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,705 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,706 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,708 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,710 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,712 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,714 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,715 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,717 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,719 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,721 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,723 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,725 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,727 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,729 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,731 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,732 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,734 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,736 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,737 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,739 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,741 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,743 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,745 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,746 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,748 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,750 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,752 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,754 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,756 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,757 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,759 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,761 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,763 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,765 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,767 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,769 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,771 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,772 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,774 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,776 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,778 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,780 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,781 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,783 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,785 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,787 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,788 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,790 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,792 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,794 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,796 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,797 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_1/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_1 2026-05-11T15:53:17,800 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,802 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,804 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,805 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,807 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,809 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,811 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,813 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,815 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,817 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,819 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,821 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,822 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,824 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,826 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,828 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,829 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,831 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,833 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,835 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,837 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,839 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,841 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,842 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,844 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,846 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,848 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,849 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,851 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,853 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,855 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,856 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,859 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,860 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,862 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,864 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,865 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,867 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,869 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,871 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,873 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,874 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,876 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,878 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,880 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,882 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,883 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,885 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,887 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,889 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,891 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,893 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,895 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,897 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,899 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,900 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,902 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,904 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,906 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_5/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_5 2026-05-11T15:53:17,908 copying build/lib/lm_eval/tasks/afrobench/sib/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib 2026-05-11T15:53:17,911 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,912 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,914 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,916 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,918 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,919 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,921 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,923 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,925 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,927 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,929 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,931 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,933 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,934 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,936 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,938 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,940 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,942 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,943 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,945 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,947 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,949 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,951 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,953 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,955 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,957 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,958 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,960 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,962 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,964 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,966 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,967 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,969 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,971 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,973 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,975 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,977 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,979 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,981 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,982 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,985 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,987 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,989 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,991 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,993 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,995 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,996 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:17,998 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,000 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,002 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,003 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,005 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,007 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,009 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,010 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,012 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,014 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,015 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,017 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_3/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_3 2026-05-11T15:53:18,021 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,022 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kmb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,024 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,026 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,029 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,030 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,032 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,034 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,036 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,037 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,039 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,041 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,043 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_nus.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,045 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,046 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_dik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,048 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,050 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,051 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,053 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,055 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_sag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,057 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,058 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,060 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kik.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,063 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tzm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,065 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_umb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,067 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,069 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_aeb.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,071 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,073 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,076 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,077 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,079 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,081 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_taq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,083 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,085 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kbp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,086 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,088 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,090 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,092 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_aka.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,093 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,095 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,097 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,099 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,101 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,102 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,104 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,106 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,108 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,110 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_knc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,111 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,113 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,115 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,117 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_lua.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,119 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,120 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_cjk.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,122 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,124 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_dyu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,126 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,128 copying build/lib/lm_eval/tasks/afrobench/sib/prompt_4/sib_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib/prompt_4 2026-05-11T15:53:18,129 copying build/lib/lm_eval/tasks/afrobench/sib/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sib 2026-05-11T15:53:18,132 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:53:18,134 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,135 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,137 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,139 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,141 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,143 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,144 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,146 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,148 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,150 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,152 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,153 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,155 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,157 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,159 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,161 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,162 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,164 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,166 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_2 2026-05-11T15:53:18,168 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,169 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,172 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,174 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,176 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,178 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,180 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,181 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,183 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,185 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,187 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,189 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,191 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,193 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,194 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,196 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,198 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,200 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,202 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_1 2026-05-11T15:53:18,204 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,205 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,207 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,209 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,211 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,213 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,215 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,216 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,219 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,221 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,222 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,224 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,226 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,227 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,229 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,231 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,232 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,234 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,236 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_5 2026-05-11T15:53:18,238 copying build/lib/lm_eval/tasks/afrobench/masakhanews/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:53:18,241 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,242 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,244 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,245 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,247 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,249 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,251 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,253 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,255 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,257 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,259 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,261 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,262 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,264 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,266 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,267 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,269 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,271 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,273 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_3 2026-05-11T15:53:18,275 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,276 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,278 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,280 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,282 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_run.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,283 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,285 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,287 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,289 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,291 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,293 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,295 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,297 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,299 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,300 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,302 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,304 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,305 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,307 copying build/lib/lm_eval/tasks/afrobench/masakhanews/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews/prompt_4 2026-05-11T15:53:18,309 copying build/lib/lm_eval/tasks/afrobench/masakhanews/masakhanews.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:53:18,311 copying build/lib/lm_eval/tasks/afrobench/masakhanews/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhanews 2026-05-11T15:53:18,314 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele 2026-05-11T15:53:18,316 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,317 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,319 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,321 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,323 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,325 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,326 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,328 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,330 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,332 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,334 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,335 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,337 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,339 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,341 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,343 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,344 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,346 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,348 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,349 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,351 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,353 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,354 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,356 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,358 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,360 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,362 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,363 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,365 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,367 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,369 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,371 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,373 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_2/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_2 2026-05-11T15:53:18,375 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,376 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,379 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,381 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,382 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,384 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,386 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,388 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,390 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,391 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,393 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,395 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,397 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,399 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,401 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,402 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,404 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,406 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,408 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,410 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,412 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,414 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,416 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,418 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,419 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,421 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,423 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,425 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,427 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,429 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,430 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,432 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,434 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_1/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_1 2026-05-11T15:53:18,436 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,437 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,439 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,441 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,443 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,445 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,447 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,449 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,450 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,452 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,454 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,456 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,458 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,460 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,461 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,463 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,465 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,467 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,469 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,471 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,473 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,474 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,476 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,478 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,479 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,481 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,483 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,485 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,487 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,489 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,491 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,492 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,495 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_5/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_5 2026-05-11T15:53:18,497 copying build/lib/lm_eval/tasks/afrobench/belebele/belebele.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele 2026-05-11T15:53:18,499 copying build/lib/lm_eval/tasks/afrobench/belebele/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele 2026-05-11T15:53:18,502 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,503 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,505 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,507 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,510 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,512 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,513 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,515 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,517 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,518 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,520 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,522 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,524 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,526 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,527 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,529 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,531 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,533 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,535 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,537 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,539 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,541 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,543 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,544 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,546 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,548 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,550 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,552 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,554 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,556 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,558 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,560 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,561 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_3/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_3 2026-05-11T15:53:18,564 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,565 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kea.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,567 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,569 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,571 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,573 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,575 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,577 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,579 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,581 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_som.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,583 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_arz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,584 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_afr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,586 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,588 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,590 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,592 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_plt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,594 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ssw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,596 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_gaz.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,598 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,600 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,602 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,604 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,605 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,607 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,609 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fuv.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,610 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,612 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,614 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,616 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,618 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,620 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,622 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,623 copying build/lib/lm_eval/tasks/afrobench/belebele/prompt_4/belebele_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele/prompt_4 2026-05-11T15:53:18,625 copying build/lib/lm_eval/tasks/afrobench/belebele/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/belebele 2026-05-11T15:53:18,628 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:53:18,630 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,631 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,633 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,635 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,637 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,639 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,641 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,643 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,645 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,646 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,648 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,650 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,652 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,654 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,656 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,658 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,660 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,661 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,663 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,665 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,667 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,669 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,671 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_2 2026-05-11T15:53:18,673 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,674 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,676 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,678 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,680 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,682 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,684 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,686 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,688 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,690 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,691 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,693 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,695 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,697 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,699 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,701 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,702 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,704 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,706 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,708 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,710 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,712 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,713 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_1 2026-05-11T15:53:18,715 copying build/lib/lm_eval/tasks/afrobench/masakhapos/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:53:18,718 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,720 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,722 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,724 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,726 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,728 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,730 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,731 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,733 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,735 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,737 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,739 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,741 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,743 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,745 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,747 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,749 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,751 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,753 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,755 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,757 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,759 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,761 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_5 2026-05-11T15:53:18,763 copying build/lib/lm_eval/tasks/afrobench/masakhapos/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:53:18,766 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,767 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,770 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,772 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,773 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,775 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,777 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,779 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,781 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,783 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,785 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,787 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,789 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,791 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,793 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,795 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,797 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,799 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,801 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,803 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,805 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,807 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,809 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_3 2026-05-11T15:53:18,811 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,813 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,815 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,817 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,819 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,821 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,823 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,824 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,826 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,828 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,830 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,832 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,834 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,835 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,837 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,839 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,841 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,843 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,845 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,847 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,849 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,851 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,853 copying build/lib/lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos/prompt_4 2026-05-11T15:53:18,855 copying build/lib/lm_eval/tasks/afrobench/masakhapos/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:53:18,857 copying build/lib/lm_eval/tasks/afrobench/masakhapos/masakhapos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhapos 2026-05-11T15:53:18,859 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt 2026-05-11T15:53:18,861 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,862 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,864 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-teo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,866 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,867 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_teo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,869 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,871 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_lug-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,873 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lgg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,875 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-nyn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,877 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,879 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,880 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_swa-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,882 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_nyn-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,884 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_lgg-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,886 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_ach-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,888 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_2/salt_ibo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_2 2026-05-11T15:53:18,891 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,892 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,894 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-teo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,896 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,898 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_teo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,900 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,902 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_lug-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,903 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lgg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,905 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-nyn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,907 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,909 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,911 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_swa-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,913 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_nyn-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,915 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_lgg-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,917 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_ach-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,919 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_1/salt_ibo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_1 2026-05-11T15:53:18,921 copying build/lib/lm_eval/tasks/afrobench/salt/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt 2026-05-11T15:53:18,923 copying build/lib/lm_eval/tasks/afrobench/salt/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt 2026-05-11T15:53:18,925 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,927 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,929 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-teo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,931 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,933 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_teo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,935 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,937 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_lug-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,939 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lgg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,941 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-nyn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,943 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,945 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ach.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,947 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_swa-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,949 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_nyn-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,951 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_lgg-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,952 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_ach-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,954 copying build/lib/lm_eval/tasks/afrobench/salt/prompt_3/salt_ibo-eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt/prompt_3 2026-05-11T15:53:18,956 copying build/lib/lm_eval/tasks/afrobench/salt/salt.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/salt 2026-05-11T15:53:18,959 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex 2026-05-11T15:53:18,960 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_2 2026-05-11T15:53:18,963 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,964 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,966 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,968 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,970 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,972 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,974 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,976 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,978 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,980 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,982 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,984 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,986 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,987 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,989 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,992 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,993 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,995 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,997 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:18,999 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,001 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,003 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,005 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,007 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,008 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,010 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,012 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,014 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,016 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,018 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,020 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,022 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,024 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,025 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/african-english 2026-05-11T15:53:19,029 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,030 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,032 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,033 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,035 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,037 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,039 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,041 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,042 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,044 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,046 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,048 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,050 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,052 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,054 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,056 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,057 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,059 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,061 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,063 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,065 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,067 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,069 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,071 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,073 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,074 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,076 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,078 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,080 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,082 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,084 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,086 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,088 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,090 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_2/english-african 2026-05-11T15:53:19,092 copying build/lib/lm_eval/tasks/afrobench/ntrex/ntrex.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex 2026-05-11T15:53:19,095 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_1 2026-05-11T15:53:19,097 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,098 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,100 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,103 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,104 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,106 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,108 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,110 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,112 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,114 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,116 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,118 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,120 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,122 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,123 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,125 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,127 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,129 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,131 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,133 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,135 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,137 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,139 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,141 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,142 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,144 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,146 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,148 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,150 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,152 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,154 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,155 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,157 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,159 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/african-english 2026-05-11T15:53:19,162 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,163 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,166 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,168 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,170 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,172 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,174 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,176 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,177 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,179 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,181 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,183 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,185 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,188 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,190 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,192 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,194 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,196 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,198 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,200 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,201 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,203 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,205 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,207 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,209 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,211 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,213 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,215 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,216 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,218 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,221 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,223 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,225 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,227 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_1/english-african 2026-05-11T15:53:19,230 copying build/lib/lm_eval/tasks/afrobench/ntrex/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex 2026-05-11T15:53:19,232 copying build/lib/lm_eval/tasks/afrobench/ntrex/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex 2026-05-11T15:53:19,235 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_3 2026-05-11T15:53:19,237 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,238 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_arb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,240 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,242 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_shi_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,244 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,246 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,248 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_orm_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,250 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,252 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,254 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_swa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,256 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,258 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,260 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,262 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,264 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,266 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ven_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,269 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mey_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,271 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,273 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,276 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,277 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tam_Taml-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,280 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,282 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ton_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,283 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_msa_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,285 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tel_Telu-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,287 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,289 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_urd_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,291 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,293 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,295 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mlg_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,297 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,299 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nde_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,301 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,303 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/african-english 2026-05-11T15:53:19,306 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,307 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,309 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mey_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,311 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,313 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-arb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,314 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,316 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nde_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,318 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,320 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,322 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ton_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,324 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,326 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,328 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,330 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-msa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,331 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,333 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-urd_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,335 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-orm_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,337 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,339 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ven_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,341 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,343 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,345 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,347 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,349 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,351 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,353 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tel_Telu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,355 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mlg_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,357 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,359 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,361 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-shi_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,363 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tam_Taml.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,364 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,366 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-swa_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,368 copying build/lib/lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/ntrex/prompt_3/english-african 2026-05-11T15:53:19,371 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores 2026-05-11T15:53:19,372 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_2 2026-05-11T15:53:19,375 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,376 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_luo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,378 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,380 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,382 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,384 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,386 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,388 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,389 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lug_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,391 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,394 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lua_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,396 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_run_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,398 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kea_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,400 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,402 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,404 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kab_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,406 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,407 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,409 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,411 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,413 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aka_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,415 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ary_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,416 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,418 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_plt_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,420 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,422 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,424 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_mos_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,426 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sun_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,428 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,430 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_swh_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,432 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,434 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,435 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_umb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,437 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,439 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,441 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,443 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ban_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,445 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,447 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,449 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,450 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sot_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,452 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,454 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,456 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,458 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,460 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nus_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,462 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,464 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,465 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,467 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,469 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,471 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tum_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,473 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_acq_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,475 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,476 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_twi_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,478 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,480 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_arz_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,482 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sag_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,484 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,486 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,487 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,489 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,491 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,493 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/african-english 2026-05-11T15:53:19,497 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,498 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,500 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lua_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,502 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,504 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,505 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aka_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,507 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,509 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,511 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,513 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-umb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,515 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,516 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,518 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,520 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-run_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,522 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,524 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,526 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,527 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,529 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sag_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,531 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,533 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,535 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,537 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,539 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,541 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,542 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,544 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,546 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,548 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,549 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,551 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,553 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,555 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,557 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,559 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,560 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,562 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,564 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,566 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-acq_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,568 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,569 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,571 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,573 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,575 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,578 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-twi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,580 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,581 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,583 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,585 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,587 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ban_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,589 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,591 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kab_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,592 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,594 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,596 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,598 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,600 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tum_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,602 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-mos_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,604 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,606 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,608 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,610 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,612 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,614 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2/english-african 2026-05-11T15:53:19,616 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_2/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_2 2026-05-11T15:53:19,619 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_1 2026-05-11T15:53:19,621 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,622 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_luo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,624 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,626 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,628 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,630 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,632 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,633 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,635 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lug_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,637 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,639 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lua_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,641 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_run_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,643 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kea_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,645 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,647 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,649 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kab_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,651 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,652 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,654 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,656 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,658 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aka_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,660 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ary_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,661 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,663 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_plt_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,665 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,667 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,669 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_mos_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,671 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sun_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,673 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,674 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_swh_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,676 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,678 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,680 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_umb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,682 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,684 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,686 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,687 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ban_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,689 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,691 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,693 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,695 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sot_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,696 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,698 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,700 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,702 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,704 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nus_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,706 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,708 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,710 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,711 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,713 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,715 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tum_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,717 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_acq_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,719 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,721 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_twi_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,723 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,725 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_arz_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,726 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sag_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,728 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,730 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,732 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,734 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,736 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,738 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/african-english 2026-05-11T15:53:19,741 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,742 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,744 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lua_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,746 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,748 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,750 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aka_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,752 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,754 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,756 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,758 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-umb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,760 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,762 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,763 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,765 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-run_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,767 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,769 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,771 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,773 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,775 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sag_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,777 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,779 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,780 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,782 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,784 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,786 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,788 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,790 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,792 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,793 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,795 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,797 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,799 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,801 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,803 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,804 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,806 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,808 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,810 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,812 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-acq_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,814 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,816 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,818 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,820 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,822 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,824 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-twi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,826 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,828 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,830 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,832 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,834 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ban_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,835 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,837 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kab_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,839 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,841 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,843 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,845 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,847 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tum_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,849 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-mos_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,851 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,853 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,855 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,857 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,859 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,861 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1/english-african 2026-05-11T15:53:19,863 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_1/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_1 2026-05-11T15:53:19,865 copying build/lib/lm_eval/tasks/afrobench/flores/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores 2026-05-11T15:53:19,867 copying build/lib/lm_eval/tasks/afrobench/flores/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores 2026-05-11T15:53:19,870 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_3 2026-05-11T15:53:19,872 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,873 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_luo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,875 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ibo_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,877 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_yor_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,879 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dyu_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,881 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_zul_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,883 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,885 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ssw_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,887 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lug_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,889 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_som_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,891 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lua_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,893 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_run_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,894 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kea_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,896 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kbp_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,899 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,900 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kab_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,902 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tsn_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,904 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,906 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tir_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,908 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,910 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aka_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,912 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ary_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,914 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kmb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,916 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_plt_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,918 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_hau_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,920 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_cjk_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,922 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_mos_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,923 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sun_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,925 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fuv_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,927 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_swh_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,929 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tzm_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,931 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,933 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_umb_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,935 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bem_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,937 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_gaz_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,939 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,941 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ban_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,943 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,945 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Tfng-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,947 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,949 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sot_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,951 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nya_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,953 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_xho_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,955 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sna_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,957 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_amh_Ethi-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,959 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nus_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,961 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tso_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,963 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kam_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,965 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lin_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,967 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ewe_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,969 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_wol_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,971 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tum_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,973 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_acq_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,975 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,976 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_twi_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,978 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aeb_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,980 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_arz_Arab-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,982 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sag_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,984 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,985 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dik_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,987 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kon_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,989 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fra_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,991 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_afr_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,993 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Latn-eng_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/african-english 2026-05-11T15:53:19,996 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:19,998 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,000 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lua_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,002 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-gaz_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,004 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,005 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aka_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,007 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,009 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-hau_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,011 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-xho_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,013 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-umb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,015 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sot_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,017 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lug_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,019 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kbp_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,020 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-run_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,022 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fra_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,024 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,026 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dyu_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,028 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,030 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sag_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,031 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,033 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,035 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tsn_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,037 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sna_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,039 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ewe_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,041 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aeb_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,043 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-som_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,045 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-luo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,047 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tso_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,048 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-wol_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,050 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ssw_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,052 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,054 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-zul_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,056 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nus_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,058 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bem_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,060 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kin_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,061 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ibo_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,063 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,065 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,067 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-acq_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,069 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-cjk_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,071 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kik_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,073 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tir_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,075 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tzm_Tfng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,077 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,079 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-twi_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,081 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,083 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ary_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,085 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nya_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,086 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kmb_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,088 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ban_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,091 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kea_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,093 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kab_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,094 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-afr_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,096 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fuv_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,098 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-plt_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,100 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bam_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,102 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tum_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,104 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-mos_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,106 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-yor_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,108 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-amh_Ethi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,110 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kon_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,112 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-swh_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,114 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sun_Latn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,116 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-arz_Arab.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3/english-african 2026-05-11T15:53:20,118 copying build/lib/lm_eval/tasks/afrobench/flores/prompt_3/flores -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores/prompt_3 2026-05-11T15:53:20,119 copying build/lib/lm_eval/tasks/afrobench/flores/flores.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/flores 2026-05-11T15:53:20,121 copying build/lib/lm_eval/tasks/afrobench/afrobench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench 2026-05-11T15:53:20,124 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent 2026-05-11T15:53:20,125 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,127 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,129 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,131 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,133 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,135 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,136 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,138 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,140 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,142 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,144 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,146 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,148 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,149 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,151 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,153 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,155 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,157 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,159 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,161 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_2 2026-05-11T15:53:20,163 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,165 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,166 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,168 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,170 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,172 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,174 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,176 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,178 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,180 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,181 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,183 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,185 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,187 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,189 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,191 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,193 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,194 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,196 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,198 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_1 2026-05-11T15:53:20,200 copying build/lib/lm_eval/tasks/afrobench/injongointent/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent 2026-05-11T15:53:20,203 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,204 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,206 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,209 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,210 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,212 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,214 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,217 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,219 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,220 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,222 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,224 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,226 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,228 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,230 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,232 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,234 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,236 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,238 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,240 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_5 2026-05-11T15:53:20,242 copying build/lib/lm_eval/tasks/afrobench/injongointent/injongointent.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent 2026-05-11T15:53:20,243 copying build/lib/lm_eval/tasks/afrobench/injongointent/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent 2026-05-11T15:53:20,246 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,247 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,249 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,251 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,253 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,255 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,257 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,259 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,261 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,263 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,265 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,267 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,269 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,271 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,273 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,275 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,278 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,279 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,281 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,283 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_3 2026-05-11T15:53:20,286 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,287 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,289 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,291 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,293 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,295 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,297 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,299 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,300 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,302 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,304 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,306 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,308 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,310 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,312 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sot.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,314 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,316 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,318 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,320 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,322 copying build/lib/lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/injongointent/prompt_4 2026-05-11T15:53:20,324 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti 2026-05-11T15:53:20,326 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,327 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,329 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,331 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,333 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,335 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,337 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,339 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_2 2026-05-11T15:53:20,341 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,342 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,344 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,346 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,348 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,350 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,351 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,353 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_1 2026-05-11T15:53:20,355 copying build/lib/lm_eval/tasks/afrobench/nollysenti/nollysenti.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti 2026-05-11T15:53:20,358 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,359 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,361 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,363 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,365 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,367 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,370 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,371 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_5 2026-05-11T15:53:20,373 copying build/lib/lm_eval/tasks/afrobench/nollysenti/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti 2026-05-11T15:53:20,376 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,377 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,379 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,381 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,383 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,386 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,388 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,390 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_3 2026-05-11T15:53:20,393 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,394 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,396 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,398 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,400 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_eng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,402 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,403 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,405 copying build/lib/lm_eval/tasks/afrobench/nollysenti/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/nollysenti/prompt_4 2026-05-11T15:53:20,408 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:53:20,410 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,411 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,413 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,415 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,417 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,419 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,421 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,423 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,425 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/run.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,427 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,430 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,432 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,434 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,436 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,438 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,440 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,442 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,444 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,445 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_2 2026-05-11T15:53:20,448 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,449 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,451 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,452 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,454 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,456 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,458 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,460 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,463 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/run.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,465 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,467 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,469 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,472 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,474 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,476 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,478 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,481 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,483 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,485 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_1 2026-05-11T15:53:20,487 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,489 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,491 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,493 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,495 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,497 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,499 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,501 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,503 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,506 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,508 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,510 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,512 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,514 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,515 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,517 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,519 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,521 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_5 2026-05-11T15:53:20,523 copying build/lib/lm_eval/tasks/afrobench/afrisenti/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:53:20,525 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,526 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,529 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,531 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,533 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,535 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,536 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,538 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,540 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,542 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,544 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,546 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,548 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,550 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,552 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,554 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,556 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,558 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_3 2026-05-11T15:53:20,560 copying build/lib/lm_eval/tasks/afrobench/afrisenti/fewshot.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:53:20,563 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,564 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_orm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,566 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,568 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,569 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,571 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_arq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,573 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,575 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,577 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,579 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tir.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,581 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,582 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,584 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/xx.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,586 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,588 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,590 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_por.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,592 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,594 copying build/lib/lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti/prompt_4 2026-05-11T15:53:20,596 copying build/lib/lm_eval/tasks/afrobench/afrisenti/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:53:20,598 copying build/lib/lm_eval/tasks/afrobench/afrisenti/afrisenti.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afrisenti 2026-05-11T15:53:20,601 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr 2026-05-11T15:53:20,603 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,604 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,606 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,608 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,610 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,611 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,613 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_2 2026-05-11T15:53:20,616 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,617 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,619 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,621 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,623 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,624 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,626 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_1 2026-05-11T15:53:20,628 copying build/lib/lm_eval/tasks/afrobench/adr/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr 2026-05-11T15:53:20,631 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,632 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,634 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,636 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,638 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,640 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,642 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_5 2026-05-11T15:53:20,643 copying build/lib/lm_eval/tasks/afrobench/adr/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr 2026-05-11T15:53:20,647 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,648 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,651 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,652 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,654 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,656 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,657 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_3 2026-05-11T15:53:20,660 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,661 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,662 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,664 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,666 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,668 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,670 copying build/lib/lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr/prompt_4 2026-05-11T15:53:20,672 copying build/lib/lm_eval/tasks/afrobench/adr/afridiacritics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/adr 2026-05-11T15:53:20,674 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum 2026-05-11T15:53:20,676 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,677 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_somali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,679 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_pidgin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,681 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_igbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,683 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_kirundi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,685 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,686 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_hausa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,688 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_tigrinya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,691 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_oromo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,693 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,694 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_amharic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,696 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_yoruba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,698 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,700 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,702 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_2 2026-05-11T15:53:20,704 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,706 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_somali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,707 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_pidgin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,709 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_igbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,711 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_kirundi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,713 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,715 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_hausa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,717 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_tigrinya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,719 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_oromo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,721 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,722 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_amharic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,724 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_yoruba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,726 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,728 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,730 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_1 2026-05-11T15:53:20,731 copying build/lib/lm_eval/tasks/afrobench/xlsum/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum 2026-05-11T15:53:20,734 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,735 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_somali.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,737 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_pidgin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,739 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_igbo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,741 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_kirundi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,743 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,745 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_hausa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,746 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_tigrinya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,748 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_oromo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,750 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_arabic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,752 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_amharic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,754 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_yoruba.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,756 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,758 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_swahili.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,759 copying build/lib/lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_telugu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum/prompt_3 2026-05-11T15:53:20,761 copying build/lib/lm_eval/tasks/afrobench/xlsum/xlsum.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum 2026-05-11T15:53:20,763 copying build/lib/lm_eval/tasks/afrobench/xlsum/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/xlsum 2026-05-11T15:53:20,766 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:53:20,767 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,768 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,770 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,772 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,774 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,776 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,777 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,779 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,781 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,783 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2 2026-05-11T15:53:20,785 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,786 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,788 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,790 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,792 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,794 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,795 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,797 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,799 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,801 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1 2026-05-11T15:53:20,804 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,805 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,807 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,809 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,811 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,812 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,814 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,816 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,818 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,820 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5 2026-05-11T15:53:20,822 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:53:20,825 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,826 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,828 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,830 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,831 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,833 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,835 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,837 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,839 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,841 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3 2026-05-11T15:53:20,842 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/uhura.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:53:20,845 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,846 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,848 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,850 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,852 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,854 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_nso.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,856 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,857 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,859 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,861 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4 2026-05-11T15:53:20,863 copying build/lib/lm_eval/tasks/afrobench/uhura-arc-easy/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/uhura-arc-easy 2026-05-11T15:53:20,866 copying build/lib/lm_eval/tasks/afrobench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench 2026-05-11T15:53:20,868 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:53:20,869 copying build/lib/lm_eval/tasks/afrobench/masakhaner/masakhaner.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:53:20,872 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,873 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,875 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,877 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,879 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,880 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,882 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,884 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,886 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,888 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,889 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,891 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,893 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,895 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,897 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,899 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,901 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,903 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,905 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,907 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,909 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,911 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,913 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_2 2026-05-11T15:53:20,915 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,916 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,918 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,919 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,921 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,923 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,925 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,927 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,928 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,930 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,932 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,934 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,936 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,938 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,940 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,942 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,944 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,946 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,948 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,950 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,952 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,953 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,956 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_1 2026-05-11T15:53:20,958 copying build/lib/lm_eval/tasks/afrobench/masakhaner/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:53:20,961 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,962 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,964 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,966 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,968 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,970 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,972 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,975 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,977 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,980 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,983 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,985 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,987 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,989 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,991 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,993 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,995 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,997 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:20,999 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:21,001 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:21,003 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:21,005 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:21,007 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_5 2026-05-11T15:53:21,009 copying build/lib/lm_eval/tasks/afrobench/masakhaner/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner 2026-05-11T15:53:21,012 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,013 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,015 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,017 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,019 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,021 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,023 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,025 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,027 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,029 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,031 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,033 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,035 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,037 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,039 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,040 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,042 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,044 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,046 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,048 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,049 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,051 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,054 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_3 2026-05-11T15:53:21,056 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,057 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,059 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_yo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,061 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,063 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,065 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_am.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,067 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,069 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ha.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,071 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,073 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_xh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,075 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,077 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_wo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,080 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ee.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,082 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_lg.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,084 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,085 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ny.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,087 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,089 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,091 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_rw.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,093 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ig.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,095 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,097 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,099 copying build/lib/lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_zu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/masakhaner/prompt_4 2026-05-11T15:53:21,102 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa 2026-05-11T15:53:21,103 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,105 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,107 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,109 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,111 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,113 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,114 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,116 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,118 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,120 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,123 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,125 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_2 2026-05-11T15:53:21,127 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,128 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,130 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,132 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,134 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,136 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,138 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,140 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,142 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,143 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,145 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,147 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_1 2026-05-11T15:53:21,150 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,151 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,153 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,155 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,157 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,159 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,161 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,163 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,165 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,167 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,169 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,171 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_5 2026-05-11T15:53:21,173 copying build/lib/lm_eval/tasks/afrobench/afriqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa 2026-05-11T15:53:21,176 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,177 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,179 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,182 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,184 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,186 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,188 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,190 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,192 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,194 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,196 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,197 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_3 2026-05-11T15:53:21,199 copying build/lib/lm_eval/tasks/afrobench/afriqa/afriqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa 2026-05-11T15:53:21,202 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,203 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,205 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,207 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,209 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,210 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_bem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,212 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,214 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,216 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,217 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,219 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,221 copying build/lib/lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa/prompt_4 2026-05-11T15:53:21,223 copying build/lib/lm_eval/tasks/afrobench/afriqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/afriqa 2026-05-11T15:53:21,225 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T15:53:21,226 copying build/lib/lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench_lite.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T15:53:21,228 copying build/lib/lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench.sh -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/sample_run_scripts 2026-05-11T15:53:21,231 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand 2026-05-11T15:53:21,234 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_2 2026-05-11T15:53:21,236 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,237 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_kin-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,239 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_pcm-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,240 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_mos-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,242 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bam-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,244 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_sna-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,246 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_lug-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,248 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_fon-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,249 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ibo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,251 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bbj-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,253 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_yor-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,255 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_xho-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,256 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ewe-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,258 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_swa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,260 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_amh-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,262 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_luo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,264 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_tsn-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,266 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_nya-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,267 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,269 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_zul-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,271 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_twi-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,273 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_wol-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,274 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,277 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_hau-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/african-english 2026-05-11T15:53:21,279 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,280 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,282 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,284 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,286 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,288 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,290 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,291 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,293 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,295 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,297 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,299 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,301 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,303 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,305 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,306 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,308 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,310 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,312 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,314 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,315 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,317 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,319 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,321 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_2/english-african/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_2/english-african 2026-05-11T15:53:21,323 copying build/lib/lm_eval/tasks/afrobench/mafand/mafand.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand 2026-05-11T15:53:21,326 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_1 2026-05-11T15:53:21,328 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,329 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_kin-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,331 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_pcm-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,333 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_mos-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,335 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bam-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,337 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_sna-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,339 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_lug-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,341 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_fon-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,343 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ibo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,345 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bbj-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,347 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_yor-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,348 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_xho-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,350 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ewe-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,352 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_swa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,354 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_amh-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,356 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_luo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,358 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_tsn-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,360 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_nya-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,361 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,363 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_zul-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,365 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_twi-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,367 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_wol-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,369 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,371 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_hau-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/african-english 2026-05-11T15:53:21,374 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,375 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,377 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,379 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,381 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,383 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,384 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,386 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,388 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,390 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,392 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,393 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,395 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,397 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,399 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,401 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,403 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,405 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,407 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,409 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,411 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,412 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,414 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,416 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_1/english-african/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_1/english-african 2026-05-11T15:53:21,419 copying build/lib/lm_eval/tasks/afrobench/mafand/gen_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand 2026-05-11T15:53:21,421 copying build/lib/lm_eval/tasks/afrobench/mafand/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand 2026-05-11T15:53:21,424 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_3 2026-05-11T15:53:21,426 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,427 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_kin-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,429 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_pcm-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,431 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_mos-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,433 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bam-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,434 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_sna-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,436 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_lug-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,438 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_fon-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,440 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ibo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,442 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bbj-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,444 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_yor-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,446 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_xho-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,448 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ewe-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,449 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_swa-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,451 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_amh-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,453 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_luo-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,454 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_tsn-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,456 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_nya-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,458 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,460 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_zul-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,462 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_twi-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,463 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_wol-fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,465 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,468 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_hau-en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/african-english 2026-05-11T15:53:21,470 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,471 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-twi.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,473 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-luo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,475 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bam.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,477 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-nya.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,479 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-sna.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,481 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-pcm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,483 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-fon.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,484 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-ibo.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,486 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-ewe.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,488 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,490 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,492 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-kin.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,494 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-hau.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,496 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-lug.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,497 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-mos.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,499 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-tsn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,501 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-xho.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,503 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-zul.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,505 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-wol.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,506 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bbj.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,508 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,510 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-amh.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,512 copying build/lib/lm_eval/tasks/afrobench/mafand/prompt_3/english-african/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/mafand/prompt_3/english-african 2026-05-11T15:53:21,515 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:53:21,517 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:53:21,519 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:53:21,521 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:53:21,523 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:53:21,525 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_2 2026-05-11T15:53:21,527 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:53:21,528 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:53:21,530 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:53:21,532 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:53:21,534 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_1 2026-05-11T15:53:21,537 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:53:21,538 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:53:21,540 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:53:21,542 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:53:21,544 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_5 2026-05-11T15:53:21,546 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/openai_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:53:21,548 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:53:21,550 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:53:21,551 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:53:21,553 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:53:21,555 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:53:21,557 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_3 2026-05-11T15:53:21,560 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:53:21,561 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_swa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:53:21,563 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_ara.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:53:21,565 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_yor.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:53:21,567 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu/prompt_4 2026-05-11T15:53:21,569 copying build/lib/lm_eval/tasks/afrobench/openai_mmlu/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench/openai_mmlu 2026-05-11T15:53:21,571 copying build/lib/lm_eval/tasks/afrobench/afrobench-lite.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/afrobench 2026-05-11T15:53:21,574 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openai-mmmlu 2026-05-11T15:53:21,575 copying build/lib/lm_eval/tasks/openai-mmmlu/languages.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T15:53:21,577 copying build/lib/lm_eval/tasks/openai-mmmlu/subjects.json -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T15:53:21,580 copying build/lib/lm_eval/tasks/openai-mmmlu/_generate_configs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T15:53:21,582 copying build/lib/lm_eval/tasks/openai-mmmlu/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu 2026-05-11T15:53:21,601 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,602 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,604 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,606 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,608 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,609 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,611 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,613 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,614 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,616 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,618 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,620 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,622 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,624 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,626 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,628 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,631 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,633 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,635 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,637 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,638 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,640 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,642 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,644 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,645 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,647 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,649 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,651 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,652 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,654 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,656 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,658 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,659 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,661 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,663 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,665 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,667 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,669 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,671 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,673 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,675 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,677 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,679 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,681 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,682 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,684 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,686 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,688 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,689 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,691 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,693 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,695 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,696 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,698 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,700 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,702 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,704 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,706 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,708 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,710 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,711 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,713 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,715 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,716 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,718 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,720 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,722 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,723 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,726 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,728 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,729 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,731 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,733 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,734 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,736 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,738 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,740 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,741 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,743 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,745 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,746 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,748 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,750 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,752 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,754 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,755 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,757 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,759 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,761 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,763 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,765 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,767 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,769 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,771 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,773 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,775 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,777 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,778 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,780 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,782 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,784 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,786 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,788 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,790 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,792 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,794 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,796 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,798 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,800 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,801 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,803 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,805 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,807 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,809 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,811 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,813 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,814 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,816 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,818 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,820 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,822 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,823 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,825 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,827 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,829 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,831 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,833 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,834 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,836 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,838 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,840 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,842 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,843 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,845 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,847 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,849 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,851 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,852 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,854 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,856 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,858 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,860 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,862 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,863 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,865 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,867 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,869 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,871 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,873 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,875 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,877 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,878 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,880 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,882 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,885 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,886 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,888 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,890 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,892 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,894 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,896 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,897 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,899 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,901 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,903 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,905 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,907 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,909 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,911 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,913 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,915 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,917 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,919 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,921 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,923 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,925 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,927 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,929 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,931 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,933 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,935 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,937 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,938 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,940 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,942 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,944 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,946 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,947 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,949 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,951 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,953 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,955 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,957 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,960 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,961 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,963 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,965 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,967 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,969 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,971 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,973 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,975 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,976 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,978 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,980 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,982 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,983 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,985 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,987 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,989 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,990 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,992 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,994 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,996 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,998 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:21,999 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,001 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,003 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,005 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,006 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,008 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,010 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,012 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,014 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,015 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,017 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,019 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,020 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,022 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,024 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,025 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,027 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,029 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,030 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,032 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,034 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,036 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,038 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,039 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,041 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,043 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,045 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,047 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,049 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,051 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,053 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,054 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,056 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,058 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,060 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,061 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,063 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,065 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,067 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,069 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,071 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,073 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,074 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,076 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,078 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,080 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,082 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,084 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,086 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,088 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,090 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,092 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,094 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,096 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,098 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,100 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,102 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,104 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,106 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,108 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,110 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,112 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,114 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,115 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,117 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,119 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,121 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,123 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,125 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,127 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,129 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,131 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,132 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,134 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,136 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,138 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,140 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,142 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,144 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,146 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,147 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,149 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,151 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,153 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,155 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,157 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,158 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,160 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,162 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,164 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,166 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,168 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,169 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,171 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,173 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,175 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,177 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,179 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,181 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,182 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,184 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,186 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,188 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,189 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,191 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,193 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,195 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,197 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,199 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,200 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,202 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,204 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,206 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,208 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,209 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,211 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,213 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,215 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,217 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,219 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,221 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,223 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,224 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,226 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,229 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,230 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,232 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,234 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,236 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,238 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,241 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,243 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,245 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,248 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,250 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,252 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,255 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,257 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,260 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,262 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,264 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,266 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,269 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,271 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,273 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,275 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,277 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,279 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,282 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,284 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,286 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,288 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,291 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,293 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,295 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,298 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,300 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,302 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,305 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,307 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,309 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,312 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,314 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,316 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,319 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,321 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,323 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,326 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,328 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,330 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,332 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,334 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,337 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,339 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,341 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,343 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,344 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,346 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,348 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,351 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,352 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,354 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,356 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,358 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,360 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,362 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,363 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,365 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,367 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,369 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,371 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,372 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,374 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,376 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,378 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,380 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,382 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,383 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,385 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,387 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,389 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,391 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,393 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,395 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,397 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,399 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,401 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,402 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,404 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,406 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,408 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,410 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,412 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,414 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,415 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,417 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,419 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,421 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,423 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,424 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,426 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,428 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,430 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,432 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,433 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,435 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,437 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,439 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,441 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,443 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,444 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,446 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,448 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,450 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,452 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,454 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,455 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,457 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,459 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,461 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,463 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,465 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,467 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,469 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,471 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,473 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,474 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,476 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,478 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,481 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,483 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,485 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,487 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,489 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,490 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,492 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,494 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,496 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,498 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,500 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,502 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,504 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,505 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,507 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,509 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,511 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,513 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,515 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,517 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,519 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,521 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,523 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,525 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,526 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,529 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,531 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,533 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,534 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,536 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,538 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,540 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,542 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,543 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,545 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,547 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,549 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,551 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,553 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,555 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,556 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,558 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,560 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,562 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,564 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,566 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,568 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,569 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,571 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,573 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,575 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,577 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,578 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,580 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,582 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,584 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,585 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,587 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,589 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,591 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,593 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,595 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,597 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,599 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,601 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,603 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,605 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,607 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,609 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,610 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,612 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,614 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,616 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,618 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,620 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,621 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,623 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,625 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,627 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,628 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,630 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,632 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,634 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,636 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,638 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,640 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,642 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,644 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,646 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,648 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,650 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,652 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,654 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,656 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,658 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,661 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,662 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,664 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,666 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,668 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,670 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,672 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,674 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,676 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,678 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,679 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,681 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,683 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,685 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,687 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,689 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,691 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,693 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,694 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,696 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,698 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,700 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,702 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,704 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,706 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,707 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,709 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,711 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,713 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,715 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,716 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,718 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,720 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,722 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,724 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,726 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,728 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,730 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,732 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,734 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,736 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,738 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,740 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,742 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,744 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,746 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,748 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,750 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,752 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,754 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,756 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,758 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,760 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,762 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,764 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,766 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,768 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,770 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,772 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,774 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,776 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,778 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,780 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,782 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,784 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,786 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,788 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,791 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,793 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,795 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,797 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,799 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,801 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,803 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,805 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,807 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,809 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,811 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,812 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,814 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,816 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,818 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,820 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,822 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,824 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,826 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,828 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,830 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,832 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,834 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,836 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,838 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,840 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,842 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,843 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,845 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,847 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,849 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,851 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,853 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,855 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,857 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,859 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,861 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,863 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,865 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,867 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,869 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,871 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,874 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,876 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,878 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,880 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,882 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,884 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,885 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,887 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,890 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,892 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,894 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,896 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,899 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,901 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,903 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,905 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,907 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,909 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,911 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,914 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,916 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,918 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,920 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,922 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,924 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,926 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,928 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,930 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,932 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,934 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,936 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,938 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,940 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,942 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,944 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,946 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,949 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,951 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,953 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,955 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,957 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,959 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,962 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,964 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,966 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,968 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,970 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,972 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,974 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,976 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,978 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,980 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,982 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,984 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,986 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,988 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,990 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,992 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,994 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,996 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:22,998 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,000 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,002 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,004 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,005 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,007 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,009 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,011 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,013 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,015 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,017 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,019 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,021 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,023 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,025 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,027 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,029 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,031 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,033 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,035 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,037 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,039 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,041 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,043 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,045 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,048 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,050 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,053 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,056 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,058 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,061 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,063 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,066 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,068 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,071 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,073 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,076 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,079 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,081 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,084 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,086 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,089 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,092 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,094 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,096 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,099 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,101 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,104 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,106 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,109 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,111 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,114 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,116 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,119 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,121 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,123 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,125 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,128 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,130 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,133 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,135 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,137 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,140 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,143 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,145 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,147 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,149 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,151 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,153 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,155 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,158 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,160 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,162 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,164 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,166 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,169 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,170 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,172 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,174 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,177 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,179 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,181 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,183 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,185 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,187 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,189 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,191 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,194 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,196 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,198 copying build/lib/lm_eval/tasks/openai-mmmlu/default/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,200 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,202 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,203 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,205 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,207 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,209 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,211 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,213 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,215 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,216 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,218 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,220 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,222 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,224 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,226 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,228 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,230 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,233 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,234 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,236 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,239 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,241 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,243 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,245 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,247 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,250 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,252 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,254 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,256 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,259 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,261 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,263 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,265 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,267 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,269 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,271 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,273 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,275 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,277 copying build/lib/lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,279 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,282 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,284 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,286 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,288 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,290 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,292 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,294 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,297 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,299 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,301 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,303 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,305 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,307 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,309 copying build/lib/lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/openai-mmmlu/default 2026-05-11T15:53:23,312 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mts_dialog 2026-05-11T15:53:23,313 copying build/lib/lm_eval/tasks/mts_dialog/mts_dialog_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T15:53:23,315 copying build/lib/lm_eval/tasks/mts_dialog/mts_dialog.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T15:53:23,317 copying build/lib/lm_eval/tasks/mts_dialog/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T15:53:23,319 copying build/lib/lm_eval/tasks/mts_dialog/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T15:53:23,321 copying build/lib/lm_eval/tasks/mts_dialog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mts_dialog 2026-05-11T15:53:23,323 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/meddialog 2026-05-11T15:53:23,324 copying build/lib/lm_eval/tasks/meddialog/meddialog_qsumm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,326 copying build/lib/lm_eval/tasks/meddialog/meddialog_raw_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,328 copying build/lib/lm_eval/tasks/meddialog/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,330 copying build/lib/lm_eval/tasks/meddialog/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,331 copying build/lib/lm_eval/tasks/meddialog/meddialog_qsumm_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,333 copying build/lib/lm_eval/tasks/meddialog/meddialog_raw_dialogues.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,335 copying build/lib/lm_eval/tasks/meddialog/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/meddialog 2026-05-11T15:53:23,338 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/cabbq 2026-05-11T15:53:23,339 copying build/lib/lm_eval/tasks/cabbq/cabbq_age.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,341 copying build/lib/lm_eval/tasks/cabbq/cabbq_nationality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,343 copying build/lib/lm_eval/tasks/cabbq/_cabbq_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,345 copying build/lib/lm_eval/tasks/cabbq/cabbq_ses.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,347 copying build/lib/lm_eval/tasks/cabbq/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,349 copying build/lib/lm_eval/tasks/cabbq/cabbq_spanish_region.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,351 copying build/lib/lm_eval/tasks/cabbq/cabbq.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,353 copying build/lib/lm_eval/tasks/cabbq/cabbq_gender.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,355 copying build/lib/lm_eval/tasks/cabbq/cabbq_lgbtqia.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,357 copying build/lib/lm_eval/tasks/cabbq/cabbq_religion.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,359 copying build/lib/lm_eval/tasks/cabbq/cabbq_physical_appearance.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,361 copying build/lib/lm_eval/tasks/cabbq/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,363 copying build/lib/lm_eval/tasks/cabbq/cabbq_race_ethnicity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,365 copying build/lib/lm_eval/tasks/cabbq/cabbq_disability_status.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/cabbq 2026-05-11T15:53:23,367 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/arithmetic 2026-05-11T15:53:23,368 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,370 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_1dc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,372 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,374 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,376 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,377 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_5ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,379 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_4ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,381 copying build/lib/lm_eval/tasks/arithmetic/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,383 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3ds.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,385 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_2dm.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,386 copying build/lib/lm_eval/tasks/arithmetic/arithmetic_3da.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/arithmetic 2026-05-11T15:53:23,389 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/babilong 2026-05-11T15:53:23,390 copying build/lib/lm_eval/tasks/babilong/babilong_qa6.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,393 copying build/lib/lm_eval/tasks/babilong/babilong_qa4.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,394 copying build/lib/lm_eval/tasks/babilong/babilong_qa15.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,396 copying build/lib/lm_eval/tasks/babilong/babilong_qa2.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,398 copying build/lib/lm_eval/tasks/babilong/babilong_qa19.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,400 copying build/lib/lm_eval/tasks/babilong/babilong.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,402 copying build/lib/lm_eval/tasks/babilong/babilong_qa1.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,404 copying build/lib/lm_eval/tasks/babilong/babilong_qa12.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,406 copying build/lib/lm_eval/tasks/babilong/babilong_qa9.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,408 copying build/lib/lm_eval/tasks/babilong/babilong_qa20.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,410 copying build/lib/lm_eval/tasks/babilong/babilong_qa8.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,412 copying build/lib/lm_eval/tasks/babilong/babilong_qa18.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,414 copying build/lib/lm_eval/tasks/babilong/babilong_qa7.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,415 copying build/lib/lm_eval/tasks/babilong/babilong_qa10.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,417 copying build/lib/lm_eval/tasks/babilong/babilong_qa11.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,419 copying build/lib/lm_eval/tasks/babilong/babilong_qa3.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,421 copying build/lib/lm_eval/tasks/babilong/_babilong_common_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,423 copying build/lib/lm_eval/tasks/babilong/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,426 copying build/lib/lm_eval/tasks/babilong/babilong_qa16.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,427 copying build/lib/lm_eval/tasks/babilong/babilong_longctx.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,430 copying build/lib/lm_eval/tasks/babilong/common_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,431 copying build/lib/lm_eval/tasks/babilong/babilong_qa17.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,433 copying build/lib/lm_eval/tasks/babilong/babilong_qa14.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,435 copying build/lib/lm_eval/tasks/babilong/babilong_qa13.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,437 copying build/lib/lm_eval/tasks/babilong/babilong_qa5.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/babilong 2026-05-11T15:53:23,440 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/careqa 2026-05-11T15:53:23,441 copying build/lib/lm_eval/tasks/careqa/careqa_open_perplexity.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,443 copying build/lib/lm_eval/tasks/careqa/careqa_open.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,445 copying build/lib/lm_eval/tasks/careqa/utils_open.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,447 copying build/lib/lm_eval/tasks/careqa/careqa_es.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,449 copying build/lib/lm_eval/tasks/careqa/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,451 copying build/lib/lm_eval/tasks/careqa/careqa_en.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,453 copying build/lib/lm_eval/tasks/careqa/utils_perplexity.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,455 copying build/lib/lm_eval/tasks/careqa/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/careqa 2026-05-11T15:53:23,457 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux-spanish 2026-05-11T15:53:23,458 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/mmlu-redux-2.0-spanish.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish 2026-05-11T15:53:23,462 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,463 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,465 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,467 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,469 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,471 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,472 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,474 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,476 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,478 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,479 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,481 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,483 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,485 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,487 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,489 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,491 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,492 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,494 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,496 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,498 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,500 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,501 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,503 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,505 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,507 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,509 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,511 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,513 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,514 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,516 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,518 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,519 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,521 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,523 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,524 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,526 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,528 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,530 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,532 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,534 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,535 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,537 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,539 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,541 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,543 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,545 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,547 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,549 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,551 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,552 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,554 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,556 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,558 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,560 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,562 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,564 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,565 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,567 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,569 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/generative/_default_template_spanish_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish/generative 2026-05-11T15:53:23,571 copying build/lib/lm_eval/tasks/mmlu-redux-spanish/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/mmlu-redux-spanish 2026-05-11T15:53:23,574 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/metabench 2026-05-11T15:53:23,575 copying build/lib/lm_eval/tasks/metabench/process_docs.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,578 copying build/lib/lm_eval/tasks/metabench/process_docs_permute.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,580 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,582 copying build/lib/lm_eval/tasks/metabench/metabench_arc_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,584 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,586 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,588 copying build/lib/lm_eval/tasks/metabench/metabench.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,590 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,592 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,594 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,596 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,598 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,600 copying build/lib/lm_eval/tasks/metabench/metabench_gsm8k_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,601 copying build/lib/lm_eval/tasks/metabench/metabench_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,603 copying build/lib/lm_eval/tasks/metabench/metabench_hellaswag_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,605 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,607 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,608 copying build/lib/lm_eval/tasks/metabench/metabench_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,610 copying build/lib/lm_eval/tasks/metabench/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,612 copying build/lib/lm_eval/tasks/metabench/metabench_arc_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,614 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande_secondary_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,616 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,618 copying build/lib/lm_eval/tasks/metabench/metabench_gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,620 copying build/lib/lm_eval/tasks/metabench/metabench_truthfulqa.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,622 copying build/lib/lm_eval/tasks/metabench/metabench_arc_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,624 copying build/lib/lm_eval/tasks/metabench/metabench_winogrande_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,626 copying build/lib/lm_eval/tasks/metabench/metabench_secondary.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,628 copying build/lib/lm_eval/tasks/metabench/metabench_arc.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,630 copying build/lib/lm_eval/tasks/metabench/metabench_mmlu_permute.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/metabench 2026-05-11T15:53:23,632 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3 2026-05-11T15:53:23,634 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct 2026-05-11T15:53:23,637 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,638 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,641 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,643 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,646 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,648 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,651 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,654 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,656 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,658 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,661 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,663 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,665 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,667 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,670 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,672 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,674 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,676 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,679 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,681 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,683 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,685 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,687 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,690 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,692 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,695 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,697 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,699 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,701 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,704 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,707 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,709 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,712 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,714 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,716 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,719 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,721 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,725 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,727 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,730 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,733 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,735 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,738 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,740 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,743 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,745 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,748 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,750 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,752 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,754 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,757 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,759 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,762 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,764 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,767 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,770 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,771 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,774 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,777 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,779 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,782 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,785 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,787 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,789 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,792 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_hi 2026-05-11T15:53:23,795 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,797 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,799 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,801 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,803 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,806 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,808 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,809 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,811 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,813 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,815 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,817 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,819 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,821 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,823 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,825 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,828 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,830 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,833 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,835 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,837 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,839 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,841 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,843 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,846 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,848 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,850 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,852 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,854 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,857 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,860 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,862 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,864 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,867 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,869 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,872 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,874 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,877 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,879 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,882 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,884 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,887 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,890 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,893 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,895 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,898 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,900 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,903 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,905 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,908 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,910 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,913 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,915 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,917 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,920 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,922 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,925 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,927 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,930 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,932 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,935 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,938 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,940 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,942 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,945 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_fr 2026-05-11T15:53:23,949 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T15:53:23,950 copying build/lib/lm_eval/tasks/llama3/instruct/arc_challenge/arc_challenge_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T15:53:23,952 copying build/lib/lm_eval/tasks/llama3/instruct/arc_challenge/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/arc_challenge 2026-05-11T15:53:23,955 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,957 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,959 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,961 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,963 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,965 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,967 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,969 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,971 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,973 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,975 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,977 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,979 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,981 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,983 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,985 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,987 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,989 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,991 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,993 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,995 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,997 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:23,999 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,001 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,003 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,006 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,008 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,010 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,012 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,013 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,015 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,017 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,019 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,022 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,024 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,026 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,029 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,031 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,033 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,035 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,037 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,039 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,041 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,043 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,045 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,047 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,049 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,051 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,053 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,055 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,057 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,058 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,060 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,062 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,064 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,066 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,068 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,070 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,072 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,074 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,076 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,078 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,080 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,082 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,084 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_it 2026-05-11T15:53:24,087 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,089 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,091 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,093 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,095 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,098 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,100 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,102 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,105 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,108 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,111 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,113 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,116 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,118 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,120 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,122 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,125 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,127 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,129 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,132 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,135 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,137 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,139 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,141 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,143 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,145 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,148 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,150 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,153 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,155 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,158 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,160 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,162 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,165 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,167 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,169 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,171 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,174 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,177 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,179 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,182 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,185 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,187 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,189 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,191 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,194 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,197 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,199 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,201 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,204 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,207 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,209 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,211 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,213 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,215 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,218 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,220 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,223 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,225 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,227 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,230 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,232 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,234 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,237 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,240 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_th 2026-05-11T15:53:24,243 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T15:53:24,244 copying build/lib/lm_eval/tasks/llama3/instruct/gsm8k/gsm8k.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/gsm8k 2026-05-11T15:53:24,248 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,249 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,251 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,254 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,255 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,258 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,260 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,262 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,263 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,265 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,267 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,270 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,271 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,274 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,276 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,279 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,281 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,283 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,285 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,287 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,289 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,291 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,293 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,296 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,298 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,300 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,302 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,304 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,306 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,308 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,310 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,312 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,314 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,316 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,318 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,320 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,322 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,324 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,326 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,328 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,330 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,332 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,334 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,336 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,338 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,340 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,342 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,344 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,346 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,348 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,350 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,352 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,354 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,356 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,358 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,360 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,362 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,365 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,367 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,370 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,372 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,374 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,375 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,378 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,380 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_de 2026-05-11T15:53:24,384 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,385 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,387 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,389 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,391 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,392 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,395 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,397 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,398 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,400 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,402 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,404 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,406 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,408 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,410 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,412 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,414 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,416 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,418 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,420 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,422 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,423 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,425 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,427 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,429 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,431 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,433 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,435 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,437 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,439 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,441 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,443 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,445 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,446 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,448 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,450 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,452 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,454 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,456 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,457 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,459 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,461 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,463 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,465 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,467 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,469 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,471 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,473 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,475 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/_mmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,477 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,478 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,480 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,482 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,484 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,486 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,488 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,490 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,491 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,493 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,495 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,497 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,499 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,501 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,503 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu 2026-05-11T15:53:24,506 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,507 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,510 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,512 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,514 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,516 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,518 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,520 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,522 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,524 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,526 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,528 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,530 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,533 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,535 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,537 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,538 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,540 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,542 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,544 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,546 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,549 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,551 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,553 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,555 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,557 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,559 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,562 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,564 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,566 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,568 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,570 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,572 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,574 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,576 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,578 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,580 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,582 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,584 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,586 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,588 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,591 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,593 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,596 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,598 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,600 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,602 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,604 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,606 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,608 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,610 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,612 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,614 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,616 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,618 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,620 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,622 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,624 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,626 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,628 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,630 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,632 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,634 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,636 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,638 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_es 2026-05-11T15:53:24,641 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,642 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,644 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_business.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,646 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/_default_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,648 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,650 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/_mmlu_pro.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,651 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,653 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_health.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,656 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,658 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,660 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,662 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_economics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,665 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,667 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,669 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,672 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,674 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,676 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_math.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pro 2026-05-11T15:53:24,681 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,682 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,685 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,687 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,690 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,692 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,694 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,697 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,699 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_continuation_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,702 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,704 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,707 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,709 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,712 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,714 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,717 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,719 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,722 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,725 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,727 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,730 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,732 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,735 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,737 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,739 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,741 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,744 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,746 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,749 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,751 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,754 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,756 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,758 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,760 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,763 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,765 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,767 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,769 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,771 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,774 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,776 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,778 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,780 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,782 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,784 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,786 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,788 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,790 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,792 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,794 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,796 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,798 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,800 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,802 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,804 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,806 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,808 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,810 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,812 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,815 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,817 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,819 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,821 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,823 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,825 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_pt 2026-05-11T15:53:24,829 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,830 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_aging.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,832 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_formal_logic.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,833 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_world_religions.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,835 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,837 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_biology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,839 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_accounting.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,841 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_government_and_politics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,843 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_security_studies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,845 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_international_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,846 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_social_sciences.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,848 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_philosophy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,850 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_management.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,852 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,854 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,856 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,858 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_macroeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,860 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_medical_genetics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,862 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_electrical_engineering.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,864 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,866 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,868 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_machine_learning.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,870 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_chemistry.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,871 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,873 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_microeconomics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,875 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_business_ethics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,877 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_disputes.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,879 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_public_relations.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,880 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_logical_fallacies.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,882 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_virology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,884 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_clinical_knowledge.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,886 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_computer_science.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,888 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_us_foreign_policy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,890 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_scenarios.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,892 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_jurisprudence.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,894 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_medicine.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,896 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_elementary_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,898 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_prehistory.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,900 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_conceptual_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,901 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_nutrition.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,903 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_mathematics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,905 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_stem.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,907 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_european_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,909 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama_template_yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,911 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_humanities.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,913 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,914 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_econometrics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,916 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_marketing.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,918 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_other.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,920 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_sociology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,922 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_law.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,924 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_world_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,926 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_abstract_algebra.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,927 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_anatomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,929 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_global_facts.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,931 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_miscellaneous.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,933 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_psychology.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,935 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_astronomy.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,937 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_sexuality.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,939 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_statistics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,940 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_physics.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,942 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_us_history.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,944 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_geography.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,946 copying build/lib/lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_computer_security.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3/instruct/mmlu_cot 2026-05-11T15:53:24,948 copying build/lib/lm_eval/tasks/llama3/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/llama3 2026-05-11T15:53:24,951 creating build/bdist.linux-armv7l/wheel/lm_eval/tasks/copal_id 2026-05-11T15:53:24,952 copying build/lib/lm_eval/tasks/copal_id/standard.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T15:53:24,954 copying build/lib/lm_eval/tasks/copal_id/README.md -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T15:53:24,956 copying build/lib/lm_eval/tasks/copal_id/colloquial.yaml -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T15:53:24,958 copying build/lib/lm_eval/tasks/copal_id/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks/copal_id 2026-05-11T15:53:24,960 copying build/lib/lm_eval/tasks/_factory.py -> build/bdist.linux-armv7l/wheel/./lm_eval/tasks 2026-05-11T15:53:24,966 creating build/bdist.linux-armv7l/wheel/lm_eval/_cli 2026-05-11T15:53:24,967 copying build/lib/lm_eval/_cli/subcommand.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,970 copying build/lib/lm_eval/_cli/validate.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,972 copying build/lib/lm_eval/_cli/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,974 copying build/lib/lm_eval/_cli/harness.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,975 copying build/lib/lm_eval/_cli/ls.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,977 copying build/lib/lm_eval/_cli/run.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,981 copying build/lib/lm_eval/_cli/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/_cli 2026-05-11T15:53:24,984 copying build/lib/lm_eval/evaluator_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:53:24,987 copying build/lib/lm_eval/result_schema.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:53:24,990 creating build/bdist.linux-armv7l/wheel/lm_eval/api 2026-05-11T15:53:24,991 copying build/lib/lm_eval/api/group.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:24,993 copying build/lib/lm_eval/api/_types.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:24,995 copying build/lib/lm_eval/api/filter.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:24,998 copying build/lib/lm_eval/api/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:25,000 copying build/lib/lm_eval/api/model.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:25,003 creating build/bdist.linux-armv7l/wheel/lm_eval/api/task 2026-05-11T15:53:25,005 copying build/lib/lm_eval/api/task/_ll.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/task 2026-05-11T15:53:25,007 copying build/lib/lm_eval/api/task/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/task 2026-05-11T15:53:25,009 copying build/lib/lm_eval/api/task/_task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/task 2026-05-11T15:53:25,012 copying build/lib/lm_eval/api/task/_generation.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/task 2026-05-11T15:53:25,014 copying build/lib/lm_eval/api/_utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:25,017 creating build/bdist.linux-armv7l/wheel/lm_eval/api/metrics 2026-05-11T15:53:25,018 copying build/lib/lm_eval/api/metrics/_types.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,021 copying build/lib/lm_eval/api/metrics/ll.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,023 copying build/lib/lm_eval/api/metrics/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,026 copying build/lib/lm_eval/api/metrics/corpus.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,029 copying build/lib/lm_eval/api/metrics/generation.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,032 copying build/lib/lm_eval/api/metrics/metric.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,034 copying build/lib/lm_eval/api/metrics/stderr.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,037 copying build/lib/lm_eval/api/metrics/reduce.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,039 copying build/lib/lm_eval/api/metrics/aggregations.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,041 copying build/lib/lm_eval/api/metrics/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,044 copying build/lib/lm_eval/api/metrics/results.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api/metrics 2026-05-11T15:53:25,047 copying build/lib/lm_eval/api/registry.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:25,050 copying build/lib/lm_eval/api/samplers.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:25,052 copying build/lib/lm_eval/api/instance.py -> build/bdist.linux-armv7l/wheel/./lm_eval/api 2026-05-11T15:53:25,055 creating build/bdist.linux-armv7l/wheel/lm_eval/config 2026-05-11T15:53:25,056 copying build/lib/lm_eval/config/group.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T15:53:25,058 copying build/lib/lm_eval/config/templates.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T15:53:25,061 creating build/bdist.linux-armv7l/wheel/lm_eval/config/formats 2026-05-11T15:53:25,062 copying build/lib/lm_eval/config/formats/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config/formats 2026-05-11T15:53:25,065 copying build/lib/lm_eval/config/formats/_base.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config/formats 2026-05-11T15:53:25,068 copying build/lib/lm_eval/config/formats/_multiple_choice.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config/formats 2026-05-11T15:53:25,070 copying build/lib/lm_eval/config/formats/_generation.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config/formats 2026-05-11T15:53:25,072 copying build/lib/lm_eval/config/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T15:53:25,074 copying build/lib/lm_eval/config/task.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T15:53:25,076 copying build/lib/lm_eval/config/evaluate_config.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T15:53:25,080 copying build/lib/lm_eval/config/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/config 2026-05-11T15:53:25,082 copying build/lib/lm_eval/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:53:25,085 creating build/bdist.linux-armv7l/wheel/lm_eval/caching 2026-05-11T15:53:25,086 copying build/lib/lm_eval/caching/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/caching 2026-05-11T15:53:25,088 copying build/lib/lm_eval/caching/cache.py -> build/bdist.linux-armv7l/wheel/./lm_eval/caching 2026-05-11T15:53:25,090 copying build/lib/lm_eval/evaluator.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:53:25,094 creating build/bdist.linux-armv7l/wheel/lm_eval/prompts 2026-05-11T15:53:25,095 copying build/lib/lm_eval/prompts/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/prompts 2026-05-11T15:53:25,098 creating build/bdist.linux-armv7l/wheel/lm_eval/scorers 2026-05-11T15:53:25,099 copying build/lib/lm_eval/scorers/_types.py -> build/bdist.linux-armv7l/wheel/./lm_eval/scorers 2026-05-11T15:53:25,101 copying build/lib/lm_eval/scorers/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/scorers 2026-05-11T15:53:25,103 copying build/lib/lm_eval/scorers/_base.py -> build/bdist.linux-armv7l/wheel/./lm_eval/scorers 2026-05-11T15:53:25,106 copying build/lib/lm_eval/scorers/extraction.py -> build/bdist.linux-armv7l/wheel/./lm_eval/scorers 2026-05-11T15:53:25,108 creating build/bdist.linux-armv7l/wheel/lm_eval/filters 2026-05-11T15:53:25,109 copying build/lib/lm_eval/filters/custom.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T15:53:25,111 copying build/lib/lm_eval/filters/decontamination.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T15:53:25,113 copying build/lib/lm_eval/filters/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T15:53:25,115 copying build/lib/lm_eval/filters/transformation.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T15:53:25,117 copying build/lib/lm_eval/filters/selection.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T15:53:25,119 copying build/lib/lm_eval/filters/extraction.py -> build/bdist.linux-armv7l/wheel/./lm_eval/filters 2026-05-11T15:53:25,122 creating build/bdist.linux-armv7l/wheel/lm_eval/loggers 2026-05-11T15:53:25,123 copying build/lib/lm_eval/loggers/wandb_logger.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T15:53:25,126 copying build/lib/lm_eval/loggers/evaluation_tracker.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T15:53:25,128 copying build/lib/lm_eval/loggers/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T15:53:25,130 copying build/lib/lm_eval/loggers/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval/loggers 2026-05-11T15:53:25,133 copying build/lib/lm_eval/defaults.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:53:25,135 copying build/lib/lm_eval/utils.py -> build/bdist.linux-armv7l/wheel/./lm_eval 2026-05-11T15:53:25,138 creating build/bdist.linux-armv7l/wheel/lm_eval/decontamination 2026-05-11T15:53:25,139 copying build/lib/lm_eval/decontamination/decontaminate.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T15:53:25,142 copying build/lib/lm_eval/decontamination/janitor.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T15:53:25,144 copying build/lib/lm_eval/decontamination/__init__.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T15:53:25,146 copying build/lib/lm_eval/decontamination/archiver.py -> build/bdist.linux-armv7l/wheel/./lm_eval/decontamination 2026-05-11T15:53:25,152 running install_egg_info 2026-05-11T15:53:25,158 Copying lm_eval.egg-info to build/bdist.linux-armv7l/wheel/./lm_eval-0.5.0.dev1-py3.11.egg-info 2026-05-11T15:53:25,186 running install_scripts 2026-05-11T15:53:25,201 creating build/bdist.linux-armv7l/wheel/lm_eval-0.5.0.dev1.dist-info/WHEEL 2026-05-11T15:53:25,204 creating '/tmp/pip-wheel-0biwgdc6/.tmp-w86uyfxv/lm_eval-0.5.0.dev1-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2026-05-11T15:53:25,207 adding 'lm_eval/__init__.py' 2026-05-11T15:53:25,208 adding 'lm_eval/__main__.py' 2026-05-11T15:53:25,210 adding 'lm_eval/defaults.py' 2026-05-11T15:53:25,213 adding 'lm_eval/evaluator.py' 2026-05-11T15:53:25,217 adding 'lm_eval/evaluator_utils.py' 2026-05-11T15:53:25,219 adding 'lm_eval/result_schema.py' 2026-05-11T15:53:25,223 adding 'lm_eval/utils.py' 2026-05-11T15:53:25,225 adding 'lm_eval/_cli/__init__.py' 2026-05-11T15:53:25,226 adding 'lm_eval/_cli/harness.py' 2026-05-11T15:53:25,228 adding 'lm_eval/_cli/ls.py' 2026-05-11T15:53:25,230 adding 'lm_eval/_cli/run.py' 2026-05-11T15:53:25,232 adding 'lm_eval/_cli/subcommand.py' 2026-05-11T15:53:25,234 adding 'lm_eval/_cli/utils.py' 2026-05-11T15:53:25,235 adding 'lm_eval/_cli/validate.py' 2026-05-11T15:53:25,237 adding 'lm_eval/api/__init__.py' 2026-05-11T15:53:25,239 adding 'lm_eval/api/_types.py' 2026-05-11T15:53:25,240 adding 'lm_eval/api/_utils.py' 2026-05-11T15:53:25,242 adding 'lm_eval/api/filter.py' 2026-05-11T15:53:25,244 adding 'lm_eval/api/group.py' 2026-05-11T15:53:25,246 adding 'lm_eval/api/instance.py' 2026-05-11T15:53:25,249 adding 'lm_eval/api/model.py' 2026-05-11T15:53:25,252 adding 'lm_eval/api/registry.py' 2026-05-11T15:53:25,254 adding 'lm_eval/api/samplers.py' 2026-05-11T15:53:25,256 adding 'lm_eval/api/metrics/__init__.py' 2026-05-11T15:53:25,257 adding 'lm_eval/api/metrics/_types.py' 2026-05-11T15:53:25,259 adding 'lm_eval/api/metrics/aggregations.py' 2026-05-11T15:53:25,261 adding 'lm_eval/api/metrics/corpus.py' 2026-05-11T15:53:25,263 adding 'lm_eval/api/metrics/generation.py' 2026-05-11T15:53:25,264 adding 'lm_eval/api/metrics/ll.py' 2026-05-11T15:53:25,266 adding 'lm_eval/api/metrics/metric.py' 2026-05-11T15:53:25,267 adding 'lm_eval/api/metrics/reduce.py' 2026-05-11T15:53:25,269 adding 'lm_eval/api/metrics/results.py' 2026-05-11T15:53:25,270 adding 'lm_eval/api/metrics/stderr.py' 2026-05-11T15:53:25,272 adding 'lm_eval/api/metrics/utils.py' 2026-05-11T15:53:25,274 adding 'lm_eval/api/task/__init__.py' 2026-05-11T15:53:25,276 adding 'lm_eval/api/task/_generation.py' 2026-05-11T15:53:25,278 adding 'lm_eval/api/task/_ll.py' 2026-05-11T15:53:25,283 adding 'lm_eval/api/task/_task.py' 2026-05-11T15:53:25,285 adding 'lm_eval/caching/__init__.py' 2026-05-11T15:53:25,286 adding 'lm_eval/caching/cache.py' 2026-05-11T15:53:25,288 adding 'lm_eval/config/__init__.py' 2026-05-11T15:53:25,291 adding 'lm_eval/config/evaluate_config.py' 2026-05-11T15:53:25,292 adding 'lm_eval/config/group.py' 2026-05-11T15:53:25,296 adding 'lm_eval/config/task.py' 2026-05-11T15:53:25,298 adding 'lm_eval/config/templates.py' 2026-05-11T15:53:25,300 adding 'lm_eval/config/utils.py' 2026-05-11T15:53:25,302 adding 'lm_eval/config/formats/__init__.py' 2026-05-11T15:53:25,305 adding 'lm_eval/config/formats/_base.py' 2026-05-11T15:53:25,306 adding 'lm_eval/config/formats/_generation.py' 2026-05-11T15:53:25,308 adding 'lm_eval/config/formats/_multiple_choice.py' 2026-05-11T15:53:25,309 adding 'lm_eval/decontamination/__init__.py' 2026-05-11T15:53:25,311 adding 'lm_eval/decontamination/archiver.py' 2026-05-11T15:53:25,313 adding 'lm_eval/decontamination/decontaminate.py' 2026-05-11T15:53:25,315 adding 'lm_eval/decontamination/janitor.py' 2026-05-11T15:53:25,317 adding 'lm_eval/filters/__init__.py' 2026-05-11T15:53:25,318 adding 'lm_eval/filters/custom.py' 2026-05-11T15:53:25,319 adding 'lm_eval/filters/decontamination.py' 2026-05-11T15:53:25,321 adding 'lm_eval/filters/extraction.py' 2026-05-11T15:53:25,322 adding 'lm_eval/filters/selection.py' 2026-05-11T15:53:25,324 adding 'lm_eval/filters/transformation.py' 2026-05-11T15:53:25,326 adding 'lm_eval/loggers/__init__.py' 2026-05-11T15:53:25,329 adding 'lm_eval/loggers/evaluation_tracker.py' 2026-05-11T15:53:25,330 adding 'lm_eval/loggers/utils.py' 2026-05-11T15:53:25,333 adding 'lm_eval/loggers/wandb_logger.py' 2026-05-11T15:53:25,335 adding 'lm_eval/models/__init__.py' 2026-05-11T15:53:25,337 adding 'lm_eval/models/anthropic_llms.py' 2026-05-11T15:53:25,341 adding 'lm_eval/models/api_models.py' 2026-05-11T15:53:25,343 adding 'lm_eval/models/dummy.py' 2026-05-11T15:53:25,345 adding 'lm_eval/models/gguf.py' 2026-05-11T15:53:25,347 adding 'lm_eval/models/hf_audiolm.py' 2026-05-11T15:53:25,348 adding 'lm_eval/models/hf_steered.py' 2026-05-11T15:53:25,353 adding 'lm_eval/models/hf_vlms.py' 2026-05-11T15:53:25,361 adding 'lm_eval/models/huggingface.py' 2026-05-11T15:53:25,363 adding 'lm_eval/models/ibm_watsonx_ai.py' 2026-05-11T15:53:25,365 adding 'lm_eval/models/mamba_lm.py' 2026-05-11T15:53:25,371 adding 'lm_eval/models/megatron_lm.py' 2026-05-11T15:53:25,373 adding 'lm_eval/models/mistral3.py' 2026-05-11T15:53:25,375 adding 'lm_eval/models/nemo_lm.py' 2026-05-11T15:53:25,379 adding 'lm_eval/models/neuron_optimum.py' 2026-05-11T15:53:25,381 adding 'lm_eval/models/openai_completions.py' 2026-05-11T15:53:25,383 adding 'lm_eval/models/optimum_ipex.py' 2026-05-11T15:53:25,384 adding 'lm_eval/models/optimum_lm.py' 2026-05-11T15:53:25,387 adding 'lm_eval/models/sglang_causallms.py' 2026-05-11T15:53:25,388 adding 'lm_eval/models/sglang_generate_API.py' 2026-05-11T15:53:25,390 adding 'lm_eval/models/textsynth.py' 2026-05-11T15:53:25,395 adding 'lm_eval/models/utils.py' 2026-05-11T15:53:25,396 adding 'lm_eval/models/utils_hf.py' 2026-05-11T15:53:25,401 adding 'lm_eval/models/vllm_causallms.py' 2026-05-11T15:53:25,403 adding 'lm_eval/models/vllm_vlms.py' 2026-05-11T15:53:25,406 adding 'lm_eval/models/winml.py' 2026-05-11T15:53:25,408 adding 'lm_eval/prompts/__init__.py' 2026-05-11T15:53:25,410 adding 'lm_eval/scorers/__init__.py' 2026-05-11T15:53:25,413 adding 'lm_eval/scorers/_base.py' 2026-05-11T15:53:25,415 adding 'lm_eval/scorers/_types.py' 2026-05-11T15:53:25,416 adding 'lm_eval/scorers/extraction.py' 2026-05-11T15:53:25,427 adding 'lm_eval/tasks/README.md' 2026-05-11T15:53:25,429 adding 'lm_eval/tasks/__init__.py' 2026-05-11T15:53:25,432 adding 'lm_eval/tasks/_factory.py' 2026-05-11T15:53:25,433 adding 'lm_eval/tasks/_index.py' 2026-05-11T15:53:25,435 adding 'lm_eval/tasks/_yaml_loader.py' 2026-05-11T15:53:25,438 adding 'lm_eval/tasks/manager.py' 2026-05-11T15:53:25,440 adding 'lm_eval/tasks/aclue/README.md' 2026-05-11T15:53:25,441 adding 'lm_eval/tasks/aclue/_aclue.yaml' 2026-05-11T15:53:25,442 adding 'lm_eval/tasks/aclue/_default_template_yaml' 2026-05-11T15:53:25,444 adding 'lm_eval/tasks/aclue/_generate_configs.py' 2026-05-11T15:53:25,445 adding 'lm_eval/tasks/aclue/aclue_ancient_chinese_culture.yaml' 2026-05-11T15:53:25,446 adding 'lm_eval/tasks/aclue/aclue_ancient_literature.yaml' 2026-05-11T15:53:25,447 adding 'lm_eval/tasks/aclue/aclue_ancient_medical.yaml' 2026-05-11T15:53:25,448 adding 'lm_eval/tasks/aclue/aclue_ancient_phonetics.yaml' 2026-05-11T15:53:25,449 adding 'lm_eval/tasks/aclue/aclue_basic_ancient_chinese.yaml' 2026-05-11T15:53:25,451 adding 'lm_eval/tasks/aclue/aclue_couplet_prediction.yaml' 2026-05-11T15:53:25,452 adding 'lm_eval/tasks/aclue/aclue_homographic_character_resolution.yaml' 2026-05-11T15:53:25,453 adding 'lm_eval/tasks/aclue/aclue_named_entity_recognition.yaml' 2026-05-11T15:53:25,454 adding 'lm_eval/tasks/aclue/aclue_poetry_appreciate.yaml' 2026-05-11T15:53:25,455 adding 'lm_eval/tasks/aclue/aclue_poetry_context_prediction.yaml' 2026-05-11T15:53:25,457 adding 'lm_eval/tasks/aclue/aclue_poetry_quality_assessment.yaml' 2026-05-11T15:53:25,458 adding 'lm_eval/tasks/aclue/aclue_poetry_sentiment_analysis.yaml' 2026-05-11T15:53:25,459 adding 'lm_eval/tasks/aclue/aclue_polysemy_resolution.yaml' 2026-05-11T15:53:25,461 adding 'lm_eval/tasks/aclue/aclue_reading_comprehension.yaml' 2026-05-11T15:53:25,462 adding 'lm_eval/tasks/aclue/aclue_sentence_segmentation.yaml' 2026-05-11T15:53:25,464 adding 'lm_eval/tasks/acpbench/README.md' 2026-05-11T15:53:25,466 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/_boolq_cot_2shot_yaml' 2026-05-11T15:53:25,467 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/act_reach.yaml' 2026-05-11T15:53:25,469 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/app.yaml' 2026-05-11T15:53:25,470 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/just.yaml' 2026-05-11T15:53:25,472 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/land.yaml' 2026-05-11T15:53:25,473 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/prog.yaml' 2026-05-11T15:53:25,475 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/reach.yaml' 2026-05-11T15:53:25,476 adding 'lm_eval/tasks/acpbench/boolq_cot_2shot/val.yaml' 2026-05-11T15:53:25,478 adding 'lm_eval/tasks/acpbench/gen_2shot/_gen_yaml_2shot' 2026-05-11T15:53:25,479 adding 'lm_eval/tasks/acpbench/gen_2shot/acp_grammar.lark' 2026-05-11T15:53:25,484 adding 'lm_eval/tasks/acpbench/gen_2shot/acp_utils.py' 2026-05-11T15:53:25,485 adding 'lm_eval/tasks/acpbench/gen_2shot/act_reach.yaml' 2026-05-11T15:53:25,487 adding 'lm_eval/tasks/acpbench/gen_2shot/app.yaml' 2026-05-11T15:53:25,488 adding 'lm_eval/tasks/acpbench/gen_2shot/just.yaml' 2026-05-11T15:53:25,490 adding 'lm_eval/tasks/acpbench/gen_2shot/land.yaml' 2026-05-11T15:53:25,491 adding 'lm_eval/tasks/acpbench/gen_2shot/next_act.yaml' 2026-05-11T15:53:25,493 adding 'lm_eval/tasks/acpbench/gen_2shot/prog.yaml' 2026-05-11T15:53:25,494 adding 'lm_eval/tasks/acpbench/gen_2shot/reach.yaml' 2026-05-11T15:53:25,496 adding 'lm_eval/tasks/acpbench/gen_2shot/val.yaml' 2026-05-11T15:53:25,497 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/_gen_yaml_2shot' 2026-05-11T15:53:25,499 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_grammar.lark' 2026-05-11T15:53:25,503 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/acp_utils.py' 2026-05-11T15:53:25,505 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/act_reach.yaml' 2026-05-11T15:53:25,507 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/app.yaml' 2026-05-11T15:53:25,509 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/just.yaml' 2026-05-11T15:53:25,511 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/land.yaml' 2026-05-11T15:53:25,513 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/next_act.yaml' 2026-05-11T15:53:25,515 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/prog.yaml' 2026-05-11T15:53:25,516 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/reach.yaml' 2026-05-11T15:53:25,518 adding 'lm_eval/tasks/acpbench/gen_2shot_with_pddl/val.yaml' 2026-05-11T15:53:25,520 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/_mcq_cot_2shot_yaml' 2026-05-11T15:53:25,521 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/act_reach.yaml' 2026-05-11T15:53:25,523 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/app.yaml' 2026-05-11T15:53:25,524 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/just.yaml' 2026-05-11T15:53:25,525 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/land.yaml' 2026-05-11T15:53:25,527 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/prog.yaml' 2026-05-11T15:53:25,528 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/reach.yaml' 2026-05-11T15:53:25,529 adding 'lm_eval/tasks/acpbench/mcq_cot_2shot/val.yaml' 2026-05-11T15:53:25,531 adding 'lm_eval/tasks/aexams/README.md' 2026-05-11T15:53:25,533 adding 'lm_eval/tasks/aexams/_aexams.yaml' 2026-05-11T15:53:25,534 adding 'lm_eval/tasks/aexams/_default_template_yaml' 2026-05-11T15:53:25,535 adding 'lm_eval/tasks/aexams/aexams_Biology.yaml' 2026-05-11T15:53:25,536 adding 'lm_eval/tasks/aexams/aexams_IslamicStudies.yaml' 2026-05-11T15:53:25,537 adding 'lm_eval/tasks/aexams/aexams_Physics.yaml' 2026-05-11T15:53:25,539 adding 'lm_eval/tasks/aexams/aexams_Science.yaml' 2026-05-11T15:53:25,540 adding 'lm_eval/tasks/aexams/aexams_Social.yaml' 2026-05-11T15:53:25,542 adding 'lm_eval/tasks/afrimgsm/README.md' 2026-05-11T15:53:25,543 adding 'lm_eval/tasks/afrimgsm/gen_utils.py' 2026-05-11T15:53:25,545 adding 'lm_eval/tasks/afrimgsm/gen_yaml.sh' 2026-05-11T15:53:25,546 adding 'lm_eval/tasks/afrimgsm/run.sh' 2026-05-11T15:53:25,548 adding 'lm_eval/tasks/afrimgsm/utils.py' 2026-05-11T15:53:25,549 adding 'lm_eval/tasks/afrimgsm/direct/afrimgsm.yaml' 2026-05-11T15:53:25,551 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_amh.yaml' 2026-05-11T15:53:25,552 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_eng.yaml' 2026-05-11T15:53:25,553 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ewe.yaml' 2026-05-11T15:53:25,555 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_fra.yaml' 2026-05-11T15:53:25,556 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_hau.yaml' 2026-05-11T15:53:25,557 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_ibo.yaml' 2026-05-11T15:53:25,558 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_kin.yaml' 2026-05-11T15:53:25,560 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lin.yaml' 2026-05-11T15:53:25,561 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_lug.yaml' 2026-05-11T15:53:25,562 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_orm.yaml' 2026-05-11T15:53:25,563 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sna.yaml' 2026-05-11T15:53:25,564 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_sot.yaml' 2026-05-11T15:53:25,566 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_swa.yaml' 2026-05-11T15:53:25,567 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_twi.yaml' 2026-05-11T15:53:25,568 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_vai.yaml' 2026-05-11T15:53:25,569 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_wol.yaml' 2026-05-11T15:53:25,570 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_xho.yaml' 2026-05-11T15:53:25,571 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yaml' 2026-05-11T15:53:25,572 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_yor.yaml' 2026-05-11T15:53:25,573 adding 'lm_eval/tasks/afrimgsm/direct/prompt_1/afrimgsm_zul.yaml' 2026-05-11T15:53:25,575 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_amh.yaml' 2026-05-11T15:53:25,576 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_eng.yaml' 2026-05-11T15:53:25,577 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ewe.yaml' 2026-05-11T15:53:25,579 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_fra.yaml' 2026-05-11T15:53:25,580 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_hau.yaml' 2026-05-11T15:53:25,581 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_ibo.yaml' 2026-05-11T15:53:25,582 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_kin.yaml' 2026-05-11T15:53:25,583 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lin.yaml' 2026-05-11T15:53:25,584 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_lug.yaml' 2026-05-11T15:53:25,586 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_orm.yaml' 2026-05-11T15:53:25,587 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sna.yaml' 2026-05-11T15:53:25,588 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_sot.yaml' 2026-05-11T15:53:25,589 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_swa.yaml' 2026-05-11T15:53:25,590 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_twi.yaml' 2026-05-11T15:53:25,591 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_vai.yaml' 2026-05-11T15:53:25,593 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_wol.yaml' 2026-05-11T15:53:25,594 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_xho.yaml' 2026-05-11T15:53:25,595 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yaml' 2026-05-11T15:53:25,596 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_yor.yaml' 2026-05-11T15:53:25,598 adding 'lm_eval/tasks/afrimgsm/direct/prompt_2/afrimgsm_zul.yaml' 2026-05-11T15:53:25,600 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_amh.yaml' 2026-05-11T15:53:25,601 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_eng.yaml' 2026-05-11T15:53:25,602 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ewe.yaml' 2026-05-11T15:53:25,603 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_fra.yaml' 2026-05-11T15:53:25,604 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_hau.yaml' 2026-05-11T15:53:25,605 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_ibo.yaml' 2026-05-11T15:53:25,607 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_kin.yaml' 2026-05-11T15:53:25,608 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lin.yaml' 2026-05-11T15:53:25,609 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_lug.yaml' 2026-05-11T15:53:25,610 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_orm.yaml' 2026-05-11T15:53:25,612 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sna.yaml' 2026-05-11T15:53:25,613 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_sot.yaml' 2026-05-11T15:53:25,614 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_swa.yaml' 2026-05-11T15:53:25,615 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_twi.yaml' 2026-05-11T15:53:25,616 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_vai.yaml' 2026-05-11T15:53:25,617 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_wol.yaml' 2026-05-11T15:53:25,619 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_xho.yaml' 2026-05-11T15:53:25,620 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yaml' 2026-05-11T15:53:25,621 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_yor.yaml' 2026-05-11T15:53:25,623 adding 'lm_eval/tasks/afrimgsm/direct/prompt_3/afrimgsm_zul.yaml' 2026-05-11T15:53:25,624 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_amh.yaml' 2026-05-11T15:53:25,626 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_eng.yaml' 2026-05-11T15:53:25,627 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ewe.yaml' 2026-05-11T15:53:25,628 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_fra.yaml' 2026-05-11T15:53:25,629 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_hau.yaml' 2026-05-11T15:53:25,630 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_ibo.yaml' 2026-05-11T15:53:25,631 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_kin.yaml' 2026-05-11T15:53:25,632 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lin.yaml' 2026-05-11T15:53:25,634 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_lug.yaml' 2026-05-11T15:53:25,635 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_orm.yaml' 2026-05-11T15:53:25,636 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sna.yaml' 2026-05-11T15:53:25,637 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_sot.yaml' 2026-05-11T15:53:25,639 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_swa.yaml' 2026-05-11T15:53:25,640 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_twi.yaml' 2026-05-11T15:53:25,641 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_vai.yaml' 2026-05-11T15:53:25,642 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_wol.yaml' 2026-05-11T15:53:25,643 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_xho.yaml' 2026-05-11T15:53:25,644 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yaml' 2026-05-11T15:53:25,645 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_yor.yaml' 2026-05-11T15:53:25,647 adding 'lm_eval/tasks/afrimgsm/direct/prompt_4/afrimgsm_zul.yaml' 2026-05-11T15:53:25,649 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_amh.yaml' 2026-05-11T15:53:25,650 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_eng.yaml' 2026-05-11T15:53:25,651 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ewe.yaml' 2026-05-11T15:53:25,652 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_fra.yaml' 2026-05-11T15:53:25,653 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_hau.yaml' 2026-05-11T15:53:25,654 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_ibo.yaml' 2026-05-11T15:53:25,656 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_kin.yaml' 2026-05-11T15:53:25,657 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lin.yaml' 2026-05-11T15:53:25,658 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_lug.yaml' 2026-05-11T15:53:25,659 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_orm.yaml' 2026-05-11T15:53:25,661 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sna.yaml' 2026-05-11T15:53:25,662 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_sot.yaml' 2026-05-11T15:53:25,663 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_swa.yaml' 2026-05-11T15:53:25,664 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_twi.yaml' 2026-05-11T15:53:25,666 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_vai.yaml' 2026-05-11T15:53:25,667 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_wol.yaml' 2026-05-11T15:53:25,668 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_xho.yaml' 2026-05-11T15:53:25,669 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yaml' 2026-05-11T15:53:25,671 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_yor.yaml' 2026-05-11T15:53:25,672 adding 'lm_eval/tasks/afrimgsm/direct/prompt_5/afrimgsm_zul.yaml' 2026-05-11T15:53:25,674 adding 'lm_eval/tasks/afrimgsm/direct_cot/afrimgsm_cot.yaml' 2026-05-11T15:53:25,676 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_amh.yaml' 2026-05-11T15:53:25,677 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_eng.yaml' 2026-05-11T15:53:25,678 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ewe.yaml' 2026-05-11T15:53:25,679 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_fra.yaml' 2026-05-11T15:53:25,680 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_hau.yaml' 2026-05-11T15:53:25,681 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_ibo.yaml' 2026-05-11T15:53:25,682 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_kin.yaml' 2026-05-11T15:53:25,684 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lin.yaml' 2026-05-11T15:53:25,685 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_lug.yaml' 2026-05-11T15:53:25,686 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_orm.yaml' 2026-05-11T15:53:25,687 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sna.yaml' 2026-05-11T15:53:25,688 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_sot.yaml' 2026-05-11T15:53:25,689 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_swa.yaml' 2026-05-11T15:53:25,690 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_twi.yaml' 2026-05-11T15:53:25,691 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_vai.yaml' 2026-05-11T15:53:25,693 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_wol.yaml' 2026-05-11T15:53:25,694 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_xho.yaml' 2026-05-11T15:53:25,695 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yaml' 2026-05-11T15:53:25,696 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_yor.yaml' 2026-05-11T15:53:25,697 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_1/afrimgsm_cot_zul.yaml' 2026-05-11T15:53:25,699 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_amh.yaml' 2026-05-11T15:53:25,700 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_eng.yaml' 2026-05-11T15:53:25,701 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ewe.yaml' 2026-05-11T15:53:25,703 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_fra.yaml' 2026-05-11T15:53:25,704 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_hau.yaml' 2026-05-11T15:53:25,705 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_ibo.yaml' 2026-05-11T15:53:25,706 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_kin.yaml' 2026-05-11T15:53:25,708 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lin.yaml' 2026-05-11T15:53:25,709 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_lug.yaml' 2026-05-11T15:53:25,710 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_orm.yaml' 2026-05-11T15:53:25,711 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sna.yaml' 2026-05-11T15:53:25,713 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_sot.yaml' 2026-05-11T15:53:25,714 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_swa.yaml' 2026-05-11T15:53:25,715 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_twi.yaml' 2026-05-11T15:53:25,716 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_vai.yaml' 2026-05-11T15:53:25,717 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_wol.yaml' 2026-05-11T15:53:25,719 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_xho.yaml' 2026-05-11T15:53:25,720 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yaml' 2026-05-11T15:53:25,722 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_yor.yaml' 2026-05-11T15:53:25,723 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_2/afrimgsm_cot_zul.yaml' 2026-05-11T15:53:25,725 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_amh.yaml' 2026-05-11T15:53:25,726 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_eng.yaml' 2026-05-11T15:53:25,727 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ewe.yaml' 2026-05-11T15:53:25,728 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_fra.yaml' 2026-05-11T15:53:25,730 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_hau.yaml' 2026-05-11T15:53:25,731 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_ibo.yaml' 2026-05-11T15:53:25,732 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_kin.yaml' 2026-05-11T15:53:25,733 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lin.yaml' 2026-05-11T15:53:25,734 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_lug.yaml' 2026-05-11T15:53:25,735 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_orm.yaml' 2026-05-11T15:53:25,736 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sna.yaml' 2026-05-11T15:53:25,738 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_sot.yaml' 2026-05-11T15:53:25,739 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_swa.yaml' 2026-05-11T15:53:25,740 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_twi.yaml' 2026-05-11T15:53:25,741 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_vai.yaml' 2026-05-11T15:53:25,742 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_wol.yaml' 2026-05-11T15:53:25,743 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_xho.yaml' 2026-05-11T15:53:25,745 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yaml' 2026-05-11T15:53:25,746 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_yor.yaml' 2026-05-11T15:53:25,747 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_3/afrimgsm_cot_zul.yaml' 2026-05-11T15:53:25,749 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_amh.yaml' 2026-05-11T15:53:25,750 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_eng.yaml' 2026-05-11T15:53:25,752 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ewe.yaml' 2026-05-11T15:53:25,753 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_fra.yaml' 2026-05-11T15:53:25,754 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_hau.yaml' 2026-05-11T15:53:25,755 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_ibo.yaml' 2026-05-11T15:53:25,757 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_kin.yaml' 2026-05-11T15:53:25,758 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lin.yaml' 2026-05-11T15:53:25,759 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_lug.yaml' 2026-05-11T15:53:25,761 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_orm.yaml' 2026-05-11T15:53:25,762 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sna.yaml' 2026-05-11T15:53:25,763 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_sot.yaml' 2026-05-11T15:53:25,764 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_swa.yaml' 2026-05-11T15:53:25,765 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_twi.yaml' 2026-05-11T15:53:25,767 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_vai.yaml' 2026-05-11T15:53:25,768 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_wol.yaml' 2026-05-11T15:53:25,769 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_xho.yaml' 2026-05-11T15:53:25,771 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yaml' 2026-05-11T15:53:25,772 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_yor.yaml' 2026-05-11T15:53:25,773 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_4/afrimgsm_cot_zul.yaml' 2026-05-11T15:53:25,775 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_amh.yaml' 2026-05-11T15:53:25,777 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_eng.yaml' 2026-05-11T15:53:25,778 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ewe.yaml' 2026-05-11T15:53:25,779 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_fra.yaml' 2026-05-11T15:53:25,780 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_hau.yaml' 2026-05-11T15:53:25,782 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_ibo.yaml' 2026-05-11T15:53:25,783 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_kin.yaml' 2026-05-11T15:53:25,784 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lin.yaml' 2026-05-11T15:53:25,785 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_lug.yaml' 2026-05-11T15:53:25,786 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_orm.yaml' 2026-05-11T15:53:25,787 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sna.yaml' 2026-05-11T15:53:25,789 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_sot.yaml' 2026-05-11T15:53:25,790 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_swa.yaml' 2026-05-11T15:53:25,791 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_twi.yaml' 2026-05-11T15:53:25,792 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_vai.yaml' 2026-05-11T15:53:25,794 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_wol.yaml' 2026-05-11T15:53:25,795 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_xho.yaml' 2026-05-11T15:53:25,796 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yaml' 2026-05-11T15:53:25,797 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_yor.yaml' 2026-05-11T15:53:25,799 adding 'lm_eval/tasks/afrimgsm/direct_cot/prompt_5/afrimgsm_cot_zul.yaml' 2026-05-11T15:53:25,800 adding 'lm_eval/tasks/afrimgsm/translate/afrimgsm_tt.yaml' 2026-05-11T15:53:25,802 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_amh.yaml' 2026-05-11T15:53:25,804 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ewe.yaml' 2026-05-11T15:53:25,805 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_fra.yaml' 2026-05-11T15:53:25,806 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_hau.yaml' 2026-05-11T15:53:25,807 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_ibo.yaml' 2026-05-11T15:53:25,808 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_kin.yaml' 2026-05-11T15:53:25,810 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lin.yaml' 2026-05-11T15:53:25,811 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_lug.yaml' 2026-05-11T15:53:25,812 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_orm.yaml' 2026-05-11T15:53:25,813 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sna.yaml' 2026-05-11T15:53:25,814 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_sot.yaml' 2026-05-11T15:53:25,815 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_swa.yaml' 2026-05-11T15:53:25,817 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_twi.yaml' 2026-05-11T15:53:25,818 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_wol.yaml' 2026-05-11T15:53:25,819 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_xho.yaml' 2026-05-11T15:53:25,820 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yaml' 2026-05-11T15:53:25,821 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_yor.yaml' 2026-05-11T15:53:25,823 adding 'lm_eval/tasks/afrimgsm/translate/prompt_1/afrimgsm_translate_zul.yaml' 2026-05-11T15:53:25,825 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_amh.yaml' 2026-05-11T15:53:25,826 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ewe.yaml' 2026-05-11T15:53:25,827 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_fra.yaml' 2026-05-11T15:53:25,828 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_hau.yaml' 2026-05-11T15:53:25,830 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_ibo.yaml' 2026-05-11T15:53:25,831 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_kin.yaml' 2026-05-11T15:53:25,832 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lin.yaml' 2026-05-11T15:53:25,833 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_lug.yaml' 2026-05-11T15:53:25,834 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_orm.yaml' 2026-05-11T15:53:25,835 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sna.yaml' 2026-05-11T15:53:25,837 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_sot.yaml' 2026-05-11T15:53:25,838 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_swa.yaml' 2026-05-11T15:53:25,839 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_twi.yaml' 2026-05-11T15:53:25,840 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_wol.yaml' 2026-05-11T15:53:25,841 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_xho.yaml' 2026-05-11T15:53:25,842 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yaml' 2026-05-11T15:53:25,844 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_yor.yaml' 2026-05-11T15:53:25,845 adding 'lm_eval/tasks/afrimgsm/translate/prompt_2/afrimgsm_translate_zul.yaml' 2026-05-11T15:53:25,847 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_amh.yaml' 2026-05-11T15:53:25,848 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ewe.yaml' 2026-05-11T15:53:25,849 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_fra.yaml' 2026-05-11T15:53:25,850 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_hau.yaml' 2026-05-11T15:53:25,851 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_ibo.yaml' 2026-05-11T15:53:25,853 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_kin.yaml' 2026-05-11T15:53:25,854 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lin.yaml' 2026-05-11T15:53:25,855 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_lug.yaml' 2026-05-11T15:53:25,856 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_orm.yaml' 2026-05-11T15:53:25,857 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sna.yaml' 2026-05-11T15:53:25,859 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_sot.yaml' 2026-05-11T15:53:25,860 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_swa.yaml' 2026-05-11T15:53:25,861 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_twi.yaml' 2026-05-11T15:53:25,862 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_wol.yaml' 2026-05-11T15:53:25,863 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_xho.yaml' 2026-05-11T15:53:25,865 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yaml' 2026-05-11T15:53:25,866 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_yor.yaml' 2026-05-11T15:53:25,867 adding 'lm_eval/tasks/afrimgsm/translate/prompt_3/afrimgsm_translate_zul.yaml' 2026-05-11T15:53:25,869 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_amh.yaml' 2026-05-11T15:53:25,870 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ewe.yaml' 2026-05-11T15:53:25,871 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_fra.yaml' 2026-05-11T15:53:25,872 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_hau.yaml' 2026-05-11T15:53:25,874 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_ibo.yaml' 2026-05-11T15:53:25,875 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_kin.yaml' 2026-05-11T15:53:25,876 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lin.yaml' 2026-05-11T15:53:25,877 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_lug.yaml' 2026-05-11T15:53:25,878 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_orm.yaml' 2026-05-11T15:53:25,880 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sna.yaml' 2026-05-11T15:53:25,881 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_sot.yaml' 2026-05-11T15:53:25,882 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_swa.yaml' 2026-05-11T15:53:25,883 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_twi.yaml' 2026-05-11T15:53:25,884 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_wol.yaml' 2026-05-11T15:53:25,885 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_xho.yaml' 2026-05-11T15:53:25,886 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yaml' 2026-05-11T15:53:25,887 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_yor.yaml' 2026-05-11T15:53:25,889 adding 'lm_eval/tasks/afrimgsm/translate/prompt_4/afrimgsm_translate_zul.yaml' 2026-05-11T15:53:25,890 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_amh.yaml' 2026-05-11T15:53:25,892 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ewe.yaml' 2026-05-11T15:53:25,893 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_fra.yaml' 2026-05-11T15:53:25,894 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_hau.yaml' 2026-05-11T15:53:25,895 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_ibo.yaml' 2026-05-11T15:53:25,896 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_kin.yaml' 2026-05-11T15:53:25,898 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lin.yaml' 2026-05-11T15:53:25,899 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_lug.yaml' 2026-05-11T15:53:25,900 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_orm.yaml' 2026-05-11T15:53:25,901 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sna.yaml' 2026-05-11T15:53:25,902 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_sot.yaml' 2026-05-11T15:53:25,903 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_swa.yaml' 2026-05-11T15:53:25,905 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_twi.yaml' 2026-05-11T15:53:25,906 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_wol.yaml' 2026-05-11T15:53:25,908 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_xho.yaml' 2026-05-11T15:53:25,909 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yaml' 2026-05-11T15:53:25,910 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_yor.yaml' 2026-05-11T15:53:25,911 adding 'lm_eval/tasks/afrimgsm/translate/prompt_5/afrimgsm_translate_zul.yaml' 2026-05-11T15:53:25,913 adding 'lm_eval/tasks/afrimgsm/translate_cot/afrimgsm_tt_cot.yaml' 2026-05-11T15:53:25,915 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_amh.yaml' 2026-05-11T15:53:25,916 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T15:53:25,918 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_fra.yaml' 2026-05-11T15:53:25,919 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_hau.yaml' 2026-05-11T15:53:25,920 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T15:53:25,921 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_kin.yaml' 2026-05-11T15:53:25,923 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lin.yaml' 2026-05-11T15:53:25,924 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_lug.yaml' 2026-05-11T15:53:25,925 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_orm.yaml' 2026-05-11T15:53:25,926 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sna.yaml' 2026-05-11T15:53:25,927 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_sot.yaml' 2026-05-11T15:53:25,928 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_swa.yaml' 2026-05-11T15:53:25,929 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_twi.yaml' 2026-05-11T15:53:25,930 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_vai.yaml' 2026-05-11T15:53:25,932 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_wol.yaml' 2026-05-11T15:53:25,933 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_xho.yaml' 2026-05-11T15:53:25,934 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yaml' 2026-05-11T15:53:25,935 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_yor.yaml' 2026-05-11T15:53:25,937 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_1/afrimgsm_cot_translate_zul.yaml' 2026-05-11T15:53:25,939 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_amh.yaml' 2026-05-11T15:53:25,940 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T15:53:25,942 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_fra.yaml' 2026-05-11T15:53:25,943 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_hau.yaml' 2026-05-11T15:53:25,944 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T15:53:25,946 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_kin.yaml' 2026-05-11T15:53:25,947 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lin.yaml' 2026-05-11T15:53:25,948 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_lug.yaml' 2026-05-11T15:53:25,949 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_orm.yaml' 2026-05-11T15:53:25,951 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sna.yaml' 2026-05-11T15:53:25,952 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_sot.yaml' 2026-05-11T15:53:25,953 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_swa.yaml' 2026-05-11T15:53:25,954 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_twi.yaml' 2026-05-11T15:53:25,956 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_vai.yaml' 2026-05-11T15:53:25,957 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_wol.yaml' 2026-05-11T15:53:25,958 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_xho.yaml' 2026-05-11T15:53:25,959 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yaml' 2026-05-11T15:53:25,961 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_yor.yaml' 2026-05-11T15:53:25,962 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_2/afrimgsm_cot_translate_zul.yaml' 2026-05-11T15:53:25,964 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_amh.yaml' 2026-05-11T15:53:25,965 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T15:53:25,967 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_fra.yaml' 2026-05-11T15:53:25,968 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_hau.yaml' 2026-05-11T15:53:25,970 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T15:53:25,971 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_kin.yaml' 2026-05-11T15:53:25,972 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lin.yaml' 2026-05-11T15:53:25,973 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_lug.yaml' 2026-05-11T15:53:25,975 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_orm.yaml' 2026-05-11T15:53:25,976 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sna.yaml' 2026-05-11T15:53:25,977 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_sot.yaml' 2026-05-11T15:53:25,978 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_swa.yaml' 2026-05-11T15:53:25,980 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_twi.yaml' 2026-05-11T15:53:25,981 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_vai.yaml' 2026-05-11T15:53:25,982 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_wol.yaml' 2026-05-11T15:53:25,983 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_xho.yaml' 2026-05-11T15:53:25,985 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yaml' 2026-05-11T15:53:25,986 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_yor.yaml' 2026-05-11T15:53:25,987 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_3/afrimgsm_cot_translate_zul.yaml' 2026-05-11T15:53:25,989 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_amh.yaml' 2026-05-11T15:53:25,990 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T15:53:25,991 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_fra.yaml' 2026-05-11T15:53:25,993 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_hau.yaml' 2026-05-11T15:53:25,994 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T15:53:25,995 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_kin.yaml' 2026-05-11T15:53:25,996 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lin.yaml' 2026-05-11T15:53:25,997 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_lug.yaml' 2026-05-11T15:53:25,999 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_orm.yaml' 2026-05-11T15:53:26,000 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sna.yaml' 2026-05-11T15:53:26,001 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_sot.yaml' 2026-05-11T15:53:26,002 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_swa.yaml' 2026-05-11T15:53:26,003 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_twi.yaml' 2026-05-11T15:53:26,005 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_vai.yaml' 2026-05-11T15:53:26,006 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_wol.yaml' 2026-05-11T15:53:26,007 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_xho.yaml' 2026-05-11T15:53:26,008 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yaml' 2026-05-11T15:53:26,009 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_yor.yaml' 2026-05-11T15:53:26,011 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_4/afrimgsm_cot_translate_zul.yaml' 2026-05-11T15:53:26,013 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_amh.yaml' 2026-05-11T15:53:26,014 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ewe.yaml' 2026-05-11T15:53:26,015 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_fra.yaml' 2026-05-11T15:53:26,016 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_hau.yaml' 2026-05-11T15:53:26,017 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_ibo.yaml' 2026-05-11T15:53:26,019 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_kin.yaml' 2026-05-11T15:53:26,020 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lin.yaml' 2026-05-11T15:53:26,021 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_lug.yaml' 2026-05-11T15:53:26,022 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_orm.yaml' 2026-05-11T15:53:26,023 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sna.yaml' 2026-05-11T15:53:26,024 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_sot.yaml' 2026-05-11T15:53:26,025 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_swa.yaml' 2026-05-11T15:53:26,027 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_twi.yaml' 2026-05-11T15:53:26,028 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_vai.yaml' 2026-05-11T15:53:26,029 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_wol.yaml' 2026-05-11T15:53:26,030 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_xho.yaml' 2026-05-11T15:53:26,031 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yaml' 2026-05-11T15:53:26,032 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_yor.yaml' 2026-05-11T15:53:26,034 adding 'lm_eval/tasks/afrimgsm/translate_cot/prompt_5/afrimgsm_cot_translate_zul.yaml' 2026-05-11T15:53:26,036 adding 'lm_eval/tasks/afrimmlu/README.md' 2026-05-11T15:53:26,037 adding 'lm_eval/tasks/afrimmlu/fewshot.sh' 2026-05-11T15:53:26,039 adding 'lm_eval/tasks/afrimmlu/gen_utils.py' 2026-05-11T15:53:26,040 adding 'lm_eval/tasks/afrimmlu/utils.py' 2026-05-11T15:53:26,042 adding 'lm_eval/tasks/afrimmlu/direct/afrimmlu.yaml' 2026-05-11T15:53:26,043 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct' 2026-05-11T15:53:26,045 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_amh.yaml' 2026-05-11T15:53:26,046 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_eng.yaml' 2026-05-11T15:53:26,047 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ewe.yaml' 2026-05-11T15:53:26,048 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_fra.yaml' 2026-05-11T15:53:26,049 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_hau.yaml' 2026-05-11T15:53:26,051 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_ibo.yaml' 2026-05-11T15:53:26,052 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_kin.yaml' 2026-05-11T15:53:26,053 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lin.yaml' 2026-05-11T15:53:26,054 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_lug.yaml' 2026-05-11T15:53:26,055 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_orm.yaml' 2026-05-11T15:53:26,056 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sna.yaml' 2026-05-11T15:53:26,058 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_sot.yaml' 2026-05-11T15:53:26,059 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_swa.yaml' 2026-05-11T15:53:26,060 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_twi.yaml' 2026-05-11T15:53:26,061 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_wol.yaml' 2026-05-11T15:53:26,062 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_xho.yaml' 2026-05-11T15:53:26,063 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_yor.yaml' 2026-05-11T15:53:26,065 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/afrimmlu_direct_zul.yaml' 2026-05-11T15:53:26,066 adding 'lm_eval/tasks/afrimmlu/direct/prompt_1/utils.py' 2026-05-11T15:53:26,068 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct' 2026-05-11T15:53:26,069 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_amh.yaml' 2026-05-11T15:53:26,070 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_eng.yaml' 2026-05-11T15:53:26,071 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ewe.yaml' 2026-05-11T15:53:26,073 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_fra.yaml' 2026-05-11T15:53:26,074 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_hau.yaml' 2026-05-11T15:53:26,075 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_ibo.yaml' 2026-05-11T15:53:26,076 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_kin.yaml' 2026-05-11T15:53:26,077 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lin.yaml' 2026-05-11T15:53:26,078 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_lug.yaml' 2026-05-11T15:53:26,079 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_orm.yaml' 2026-05-11T15:53:26,080 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sna.yaml' 2026-05-11T15:53:26,081 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_sot.yaml' 2026-05-11T15:53:26,082 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_swa.yaml' 2026-05-11T15:53:26,083 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_twi.yaml' 2026-05-11T15:53:26,085 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_wol.yaml' 2026-05-11T15:53:26,086 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_xho.yaml' 2026-05-11T15:53:26,087 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_yor.yaml' 2026-05-11T15:53:26,088 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/afrimmlu_direct_zul.yaml' 2026-05-11T15:53:26,089 adding 'lm_eval/tasks/afrimmlu/direct/prompt_2/utils.py' 2026-05-11T15:53:26,091 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct' 2026-05-11T15:53:26,092 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_amh.yaml' 2026-05-11T15:53:26,094 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_eng.yaml' 2026-05-11T15:53:26,095 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ewe.yaml' 2026-05-11T15:53:26,096 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_fra.yaml' 2026-05-11T15:53:26,097 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_hau.yaml' 2026-05-11T15:53:26,098 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_ibo.yaml' 2026-05-11T15:53:26,099 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_kin.yaml' 2026-05-11T15:53:26,101 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lin.yaml' 2026-05-11T15:53:26,102 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_lug.yaml' 2026-05-11T15:53:26,103 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_orm.yaml' 2026-05-11T15:53:26,104 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sna.yaml' 2026-05-11T15:53:26,105 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_sot.yaml' 2026-05-11T15:53:26,106 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_swa.yaml' 2026-05-11T15:53:26,107 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_twi.yaml' 2026-05-11T15:53:26,108 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_wol.yaml' 2026-05-11T15:53:26,110 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_xho.yaml' 2026-05-11T15:53:26,111 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_yor.yaml' 2026-05-11T15:53:26,112 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/afrimmlu_direct_zul.yaml' 2026-05-11T15:53:26,113 adding 'lm_eval/tasks/afrimmlu/direct/prompt_3/utils.py' 2026-05-11T15:53:26,115 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct' 2026-05-11T15:53:26,116 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_amh.yaml' 2026-05-11T15:53:26,117 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_eng.yaml' 2026-05-11T15:53:26,119 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ewe.yaml' 2026-05-11T15:53:26,120 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_fra.yaml' 2026-05-11T15:53:26,121 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_hau.yaml' 2026-05-11T15:53:26,122 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_ibo.yaml' 2026-05-11T15:53:26,123 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_kin.yaml' 2026-05-11T15:53:26,124 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lin.yaml' 2026-05-11T15:53:26,125 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_lug.yaml' 2026-05-11T15:53:26,126 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_orm.yaml' 2026-05-11T15:53:26,128 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sna.yaml' 2026-05-11T15:53:26,129 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_sot.yaml' 2026-05-11T15:53:26,130 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_swa.yaml' 2026-05-11T15:53:26,131 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_twi.yaml' 2026-05-11T15:53:26,132 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_wol.yaml' 2026-05-11T15:53:26,133 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_xho.yaml' 2026-05-11T15:53:26,135 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_yor.yaml' 2026-05-11T15:53:26,136 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/afrimmlu_direct_zul.yaml' 2026-05-11T15:53:26,137 adding 'lm_eval/tasks/afrimmlu/direct/prompt_4/utils.py' 2026-05-11T15:53:26,139 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct' 2026-05-11T15:53:26,140 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_amh.yaml' 2026-05-11T15:53:26,141 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_eng.yaml' 2026-05-11T15:53:26,142 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ewe.yaml' 2026-05-11T15:53:26,144 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_fra.yaml' 2026-05-11T15:53:26,145 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_hau.yaml' 2026-05-11T15:53:26,146 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_ibo.yaml' 2026-05-11T15:53:26,147 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_kin.yaml' 2026-05-11T15:53:26,148 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lin.yaml' 2026-05-11T15:53:26,149 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_lug.yaml' 2026-05-11T15:53:26,150 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_orm.yaml' 2026-05-11T15:53:26,152 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sna.yaml' 2026-05-11T15:53:26,153 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_sot.yaml' 2026-05-11T15:53:26,154 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_swa.yaml' 2026-05-11T15:53:26,155 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_twi.yaml' 2026-05-11T15:53:26,156 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_wol.yaml' 2026-05-11T15:53:26,157 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_xho.yaml' 2026-05-11T15:53:26,159 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_yor.yaml' 2026-05-11T15:53:26,160 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/afrimmlu_direct_zul.yaml' 2026-05-11T15:53:26,161 adding 'lm_eval/tasks/afrimmlu/direct/prompt_5/utils.py' 2026-05-11T15:53:26,163 adding 'lm_eval/tasks/afrimmlu/translate/afrimmlu_tt.yaml' 2026-05-11T15:53:26,165 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate' 2026-05-11T15:53:26,166 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_amh.yaml' 2026-05-11T15:53:26,167 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ewe.yaml' 2026-05-11T15:53:26,168 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_fra.yaml' 2026-05-11T15:53:26,169 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_hau.yaml' 2026-05-11T15:53:26,171 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_ibo.yaml' 2026-05-11T15:53:26,172 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_kin.yaml' 2026-05-11T15:53:26,173 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lin.yaml' 2026-05-11T15:53:26,174 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_lug.yaml' 2026-05-11T15:53:26,175 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_orm.yaml' 2026-05-11T15:53:26,176 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sna.yaml' 2026-05-11T15:53:26,177 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_sot.yaml' 2026-05-11T15:53:26,178 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_swa.yaml' 2026-05-11T15:53:26,180 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_twi.yaml' 2026-05-11T15:53:26,181 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_wol.yaml' 2026-05-11T15:53:26,182 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_xho.yaml' 2026-05-11T15:53:26,183 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_yor.yaml' 2026-05-11T15:53:26,184 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/afrimmlu_translate_zul.yaml' 2026-05-11T15:53:26,185 adding 'lm_eval/tasks/afrimmlu/translate/prompt_1/utils.py' 2026-05-11T15:53:26,187 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate' 2026-05-11T15:53:26,189 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_amh.yaml' 2026-05-11T15:53:26,190 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ewe.yaml' 2026-05-11T15:53:26,191 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_fra.yaml' 2026-05-11T15:53:26,192 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_hau.yaml' 2026-05-11T15:53:26,193 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_ibo.yaml' 2026-05-11T15:53:26,195 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_kin.yaml' 2026-05-11T15:53:26,196 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lin.yaml' 2026-05-11T15:53:26,197 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_lug.yaml' 2026-05-11T15:53:26,198 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_orm.yaml' 2026-05-11T15:53:26,199 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sna.yaml' 2026-05-11T15:53:26,201 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_sot.yaml' 2026-05-11T15:53:26,202 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_swa.yaml' 2026-05-11T15:53:26,203 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_twi.yaml' 2026-05-11T15:53:26,204 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_wol.yaml' 2026-05-11T15:53:26,205 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_xho.yaml' 2026-05-11T15:53:26,206 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_yor.yaml' 2026-05-11T15:53:26,207 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/afrimmlu_translate_zul.yaml' 2026-05-11T15:53:26,209 adding 'lm_eval/tasks/afrimmlu/translate/prompt_2/utils.py' 2026-05-11T15:53:26,211 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate' 2026-05-11T15:53:26,212 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_amh.yaml' 2026-05-11T15:53:26,213 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ewe.yaml' 2026-05-11T15:53:26,214 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_fra.yaml' 2026-05-11T15:53:26,216 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_hau.yaml' 2026-05-11T15:53:26,217 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_ibo.yaml' 2026-05-11T15:53:26,218 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_kin.yaml' 2026-05-11T15:53:26,219 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lin.yaml' 2026-05-11T15:53:26,220 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_lug.yaml' 2026-05-11T15:53:26,221 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_orm.yaml' 2026-05-11T15:53:26,222 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sna.yaml' 2026-05-11T15:53:26,223 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_sot.yaml' 2026-05-11T15:53:26,224 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_swa.yaml' 2026-05-11T15:53:26,226 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_twi.yaml' 2026-05-11T15:53:26,227 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_wol.yaml' 2026-05-11T15:53:26,228 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_xho.yaml' 2026-05-11T15:53:26,229 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_yor.yaml' 2026-05-11T15:53:26,230 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/afrimmlu_translate_zul.yaml' 2026-05-11T15:53:26,231 adding 'lm_eval/tasks/afrimmlu/translate/prompt_3/utils.py' 2026-05-11T15:53:26,233 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate' 2026-05-11T15:53:26,235 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_amh.yaml' 2026-05-11T15:53:26,236 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ewe.yaml' 2026-05-11T15:53:26,237 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_fra.yaml' 2026-05-11T15:53:26,238 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_hau.yaml' 2026-05-11T15:53:26,239 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_ibo.yaml' 2026-05-11T15:53:26,240 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_kin.yaml' 2026-05-11T15:53:26,242 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lin.yaml' 2026-05-11T15:53:26,243 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_lug.yaml' 2026-05-11T15:53:26,244 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_orm.yaml' 2026-05-11T15:53:26,245 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sna.yaml' 2026-05-11T15:53:26,246 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_sot.yaml' 2026-05-11T15:53:26,247 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_swa.yaml' 2026-05-11T15:53:26,248 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_twi.yaml' 2026-05-11T15:53:26,250 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_wol.yaml' 2026-05-11T15:53:26,251 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_xho.yaml' 2026-05-11T15:53:26,252 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_yor.yaml' 2026-05-11T15:53:26,253 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/afrimmlu_translate_zul.yaml' 2026-05-11T15:53:26,254 adding 'lm_eval/tasks/afrimmlu/translate/prompt_4/utils.py' 2026-05-11T15:53:26,256 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate' 2026-05-11T15:53:26,257 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_amh.yaml' 2026-05-11T15:53:26,259 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ewe.yaml' 2026-05-11T15:53:26,260 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_fra.yaml' 2026-05-11T15:53:26,261 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_hau.yaml' 2026-05-11T15:53:26,262 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_ibo.yaml' 2026-05-11T15:53:26,263 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_kin.yaml' 2026-05-11T15:53:26,264 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lin.yaml' 2026-05-11T15:53:26,265 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_lug.yaml' 2026-05-11T15:53:26,267 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_orm.yaml' 2026-05-11T15:53:26,268 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sna.yaml' 2026-05-11T15:53:26,269 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_sot.yaml' 2026-05-11T15:53:26,270 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_swa.yaml' 2026-05-11T15:53:26,271 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_twi.yaml' 2026-05-11T15:53:26,272 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_wol.yaml' 2026-05-11T15:53:26,273 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_xho.yaml' 2026-05-11T15:53:26,274 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_yor.yaml' 2026-05-11T15:53:26,275 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/afrimmlu_translate_zul.yaml' 2026-05-11T15:53:26,276 adding 'lm_eval/tasks/afrimmlu/translate/prompt_5/utils.py' 2026-05-11T15:53:26,278 adding 'lm_eval/tasks/afrixnli/README.md' 2026-05-11T15:53:26,280 adding 'lm_eval/tasks/afrixnli/gen_utils.py' 2026-05-11T15:53:26,281 adding 'lm_eval/tasks/afrixnli/utils.py' 2026-05-11T15:53:26,284 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_amh.yaml' 2026-05-11T15:53:26,285 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_eng.yaml' 2026-05-11T15:53:26,286 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ewe.yaml' 2026-05-11T15:53:26,287 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_fra.yaml' 2026-05-11T15:53:26,288 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_hau.yaml' 2026-05-11T15:53:26,289 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_ibo.yaml' 2026-05-11T15:53:26,290 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_kin.yaml' 2026-05-11T15:53:26,292 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lin.yaml' 2026-05-11T15:53:26,293 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_lug.yaml' 2026-05-11T15:53:26,294 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_orm.yaml' 2026-05-11T15:53:26,295 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sna.yaml' 2026-05-11T15:53:26,296 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_sot.yaml' 2026-05-11T15:53:26,297 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_swa.yaml' 2026-05-11T15:53:26,298 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_twi.yaml' 2026-05-11T15:53:26,300 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_wol.yaml' 2026-05-11T15:53:26,301 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_xho.yaml' 2026-05-11T15:53:26,302 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yaml' 2026-05-11T15:53:26,303 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_yor.yaml' 2026-05-11T15:53:26,304 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/afrixnli_en_direct_zul.yaml' 2026-05-11T15:53:26,305 adding 'lm_eval/tasks/afrixnli/anli prompt/en-direct/utils.py' 2026-05-11T15:53:26,307 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_amh.yaml' 2026-05-11T15:53:26,308 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_eng.yaml' 2026-05-11T15:53:26,309 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ewe.yaml' 2026-05-11T15:53:26,310 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_fra.yaml' 2026-05-11T15:53:26,312 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_hau.yaml' 2026-05-11T15:53:26,313 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_ibo.yaml' 2026-05-11T15:53:26,314 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_kin.yaml' 2026-05-11T15:53:26,315 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lin.yaml' 2026-05-11T15:53:26,316 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_lug.yaml' 2026-05-11T15:53:26,317 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_orm.yaml' 2026-05-11T15:53:26,319 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sna.yaml' 2026-05-11T15:53:26,320 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_sot.yaml' 2026-05-11T15:53:26,321 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_swa.yaml' 2026-05-11T15:53:26,322 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_twi.yaml' 2026-05-11T15:53:26,323 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_wol.yaml' 2026-05-11T15:53:26,324 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_xho.yaml' 2026-05-11T15:53:26,325 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yaml' 2026-05-11T15:53:26,326 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_yor.yaml' 2026-05-11T15:53:26,328 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/afrixnli_native_direct_zul.yaml' 2026-05-11T15:53:26,329 adding 'lm_eval/tasks/afrixnli/anli prompt/native-direct/utils.py' 2026-05-11T15:53:26,331 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_amh.yaml' 2026-05-11T15:53:26,332 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ewe.yaml' 2026-05-11T15:53:26,333 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_fra.yaml' 2026-05-11T15:53:26,334 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_hau.yaml' 2026-05-11T15:53:26,335 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_ibo.yaml' 2026-05-11T15:53:26,336 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_kin.yaml' 2026-05-11T15:53:26,337 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lin.yaml' 2026-05-11T15:53:26,338 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_lug.yaml' 2026-05-11T15:53:26,340 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_orm.yaml' 2026-05-11T15:53:26,341 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sna.yaml' 2026-05-11T15:53:26,342 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_sot.yaml' 2026-05-11T15:53:26,343 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_swa.yaml' 2026-05-11T15:53:26,344 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_twi.yaml' 2026-05-11T15:53:26,345 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_wol.yaml' 2026-05-11T15:53:26,346 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_xho.yaml' 2026-05-11T15:53:26,348 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yaml' 2026-05-11T15:53:26,349 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_yor.yaml' 2026-05-11T15:53:26,350 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/afrixnli_translate_zul.yaml' 2026-05-11T15:53:26,351 adding 'lm_eval/tasks/afrixnli/anli prompt/translate/utils.py' 2026-05-11T15:53:26,353 adding 'lm_eval/tasks/afrixnli/direct/afrixnli.yaml' 2026-05-11T15:53:26,355 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_amh.yaml' 2026-05-11T15:53:26,356 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_eng.yaml' 2026-05-11T15:53:26,357 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ewe.yaml' 2026-05-11T15:53:26,359 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_fra.yaml' 2026-05-11T15:53:26,360 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_hau.yaml' 2026-05-11T15:53:26,361 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_ibo.yaml' 2026-05-11T15:53:26,362 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_kin.yaml' 2026-05-11T15:53:26,363 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lin.yaml' 2026-05-11T15:53:26,365 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_lug.yaml' 2026-05-11T15:53:26,366 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_orm.yaml' 2026-05-11T15:53:26,367 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sna.yaml' 2026-05-11T15:53:26,368 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_sot.yaml' 2026-05-11T15:53:26,369 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_swa.yaml' 2026-05-11T15:53:26,370 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_twi.yaml' 2026-05-11T15:53:26,371 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_wol.yaml' 2026-05-11T15:53:26,373 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_xho.yaml' 2026-05-11T15:53:26,374 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yaml' 2026-05-11T15:53:26,375 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_yor.yaml' 2026-05-11T15:53:26,376 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/afrixnli_zul.yaml' 2026-05-11T15:53:26,377 adding 'lm_eval/tasks/afrixnli/direct/prompt_1/utils.py' 2026-05-11T15:53:26,379 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_amh.yaml' 2026-05-11T15:53:26,380 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_eng.yaml' 2026-05-11T15:53:26,381 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ewe.yaml' 2026-05-11T15:53:26,382 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_fra.yaml' 2026-05-11T15:53:26,383 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_hau.yaml' 2026-05-11T15:53:26,384 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_ibo.yaml' 2026-05-11T15:53:26,392 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_kin.yaml' 2026-05-11T15:53:26,394 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lin.yaml' 2026-05-11T15:53:26,395 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_lug.yaml' 2026-05-11T15:53:26,396 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_orm.yaml' 2026-05-11T15:53:26,397 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sna.yaml' 2026-05-11T15:53:26,398 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_sot.yaml' 2026-05-11T15:53:26,399 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_swa.yaml' 2026-05-11T15:53:26,401 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_twi.yaml' 2026-05-11T15:53:26,402 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_wol.yaml' 2026-05-11T15:53:26,403 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_xho.yaml' 2026-05-11T15:53:26,404 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yaml' 2026-05-11T15:53:26,405 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_yor.yaml' 2026-05-11T15:53:26,407 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/afrixnli_zul.yaml' 2026-05-11T15:53:26,408 adding 'lm_eval/tasks/afrixnli/direct/prompt_2/utils.py' 2026-05-11T15:53:26,410 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_amh.yaml' 2026-05-11T15:53:26,411 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_eng.yaml' 2026-05-11T15:53:26,412 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ewe.yaml' 2026-05-11T15:53:26,414 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_fra.yaml' 2026-05-11T15:53:26,415 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_hau.yaml' 2026-05-11T15:53:26,416 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_ibo.yaml' 2026-05-11T15:53:26,417 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_kin.yaml' 2026-05-11T15:53:26,418 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lin.yaml' 2026-05-11T15:53:26,420 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_lug.yaml' 2026-05-11T15:53:26,421 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_orm.yaml' 2026-05-11T15:53:26,422 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sna.yaml' 2026-05-11T15:53:26,423 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_sot.yaml' 2026-05-11T15:53:26,424 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_swa.yaml' 2026-05-11T15:53:26,425 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_twi.yaml' 2026-05-11T15:53:26,426 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_wol.yaml' 2026-05-11T15:53:26,428 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_xho.yaml' 2026-05-11T15:53:26,429 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yaml' 2026-05-11T15:53:26,430 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_yor.yaml' 2026-05-11T15:53:26,431 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/afrixnli_zul.yaml' 2026-05-11T15:53:26,432 adding 'lm_eval/tasks/afrixnli/direct/prompt_3/utils.py' 2026-05-11T15:53:26,434 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_amh.yaml' 2026-05-11T15:53:26,436 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_eng.yaml' 2026-05-11T15:53:26,437 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ewe.yaml' 2026-05-11T15:53:26,438 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_fra.yaml' 2026-05-11T15:53:26,439 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_hau.yaml' 2026-05-11T15:53:26,441 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_ibo.yaml' 2026-05-11T15:53:26,442 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_kin.yaml' 2026-05-11T15:53:26,443 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lin.yaml' 2026-05-11T15:53:26,444 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_lug.yaml' 2026-05-11T15:53:26,445 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_orm.yaml' 2026-05-11T15:53:26,446 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sna.yaml' 2026-05-11T15:53:26,447 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_sot.yaml' 2026-05-11T15:53:26,449 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_swa.yaml' 2026-05-11T15:53:26,450 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_twi.yaml' 2026-05-11T15:53:26,451 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_wol.yaml' 2026-05-11T15:53:26,452 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_xho.yaml' 2026-05-11T15:53:26,453 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yaml' 2026-05-11T15:53:26,455 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_yor.yaml' 2026-05-11T15:53:26,456 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/afrixnli_zul.yaml' 2026-05-11T15:53:26,457 adding 'lm_eval/tasks/afrixnli/direct/prompt_4/utils.py' 2026-05-11T15:53:26,459 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_amh.yaml' 2026-05-11T15:53:26,460 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_eng.yaml' 2026-05-11T15:53:26,462 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ewe.yaml' 2026-05-11T15:53:26,463 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_fra.yaml' 2026-05-11T15:53:26,464 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_hau.yaml' 2026-05-11T15:53:26,465 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_ibo.yaml' 2026-05-11T15:53:26,466 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_kin.yaml' 2026-05-11T15:53:26,468 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lin.yaml' 2026-05-11T15:53:26,469 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_lug.yaml' 2026-05-11T15:53:26,470 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_orm.yaml' 2026-05-11T15:53:26,471 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sna.yaml' 2026-05-11T15:53:26,473 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_sot.yaml' 2026-05-11T15:53:26,474 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_swa.yaml' 2026-05-11T15:53:26,475 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_twi.yaml' 2026-05-11T15:53:26,476 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_wol.yaml' 2026-05-11T15:53:26,478 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_xho.yaml' 2026-05-11T15:53:26,479 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yaml' 2026-05-11T15:53:26,480 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_yor.yaml' 2026-05-11T15:53:26,481 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/afrixnli_zul.yaml' 2026-05-11T15:53:26,482 adding 'lm_eval/tasks/afrixnli/direct/prompt_5/utils.py' 2026-05-11T15:53:26,485 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_amh.yaml' 2026-05-11T15:53:26,486 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_eng.yaml' 2026-05-11T15:53:26,487 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ewe.yaml' 2026-05-11T15:53:26,488 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_fra.yaml' 2026-05-11T15:53:26,489 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_hau.yaml' 2026-05-11T15:53:26,490 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_ibo.yaml' 2026-05-11T15:53:26,491 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_kin.yaml' 2026-05-11T15:53:26,492 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lin.yaml' 2026-05-11T15:53:26,493 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_lug.yaml' 2026-05-11T15:53:26,495 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_orm.yaml' 2026-05-11T15:53:26,496 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sna.yaml' 2026-05-11T15:53:26,497 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_sot.yaml' 2026-05-11T15:53:26,498 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_swa.yaml' 2026-05-11T15:53:26,499 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_twi.yaml' 2026-05-11T15:53:26,500 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_wol.yaml' 2026-05-11T15:53:26,501 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_xho.yaml' 2026-05-11T15:53:26,503 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yaml' 2026-05-11T15:53:26,504 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_yor.yaml' 2026-05-11T15:53:26,505 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/afrixnli_manual_direct_zul.yaml' 2026-05-11T15:53:26,506 adding 'lm_eval/tasks/afrixnli/lai prompt/direct/utils.py' 2026-05-11T15:53:26,508 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_amh.yaml' 2026-05-11T15:53:26,509 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ewe.yaml' 2026-05-11T15:53:26,510 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_fra.yaml' 2026-05-11T15:53:26,512 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_hau.yaml' 2026-05-11T15:53:26,513 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_ibo.yaml' 2026-05-11T15:53:26,514 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_kin.yaml' 2026-05-11T15:53:26,515 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lin.yaml' 2026-05-11T15:53:26,516 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_lug.yaml' 2026-05-11T15:53:26,517 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_orm.yaml' 2026-05-11T15:53:26,519 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sna.yaml' 2026-05-11T15:53:26,520 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_sot.yaml' 2026-05-11T15:53:26,521 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_swa.yaml' 2026-05-11T15:53:26,522 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_twi.yaml' 2026-05-11T15:53:26,524 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_wol.yaml' 2026-05-11T15:53:26,525 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_xho.yaml' 2026-05-11T15:53:26,526 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yaml' 2026-05-11T15:53:26,527 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_yor.yaml' 2026-05-11T15:53:26,528 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/afrixnli_manual_translate_zul.yaml' 2026-05-11T15:53:26,529 adding 'lm_eval/tasks/afrixnli/lai prompt/translate/utils.py' 2026-05-11T15:53:26,531 adding 'lm_eval/tasks/afrixnli/translate/afrixnli_tt.yaml' 2026-05-11T15:53:26,533 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_amh.yaml' 2026-05-11T15:53:26,534 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ewe.yaml' 2026-05-11T15:53:26,535 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_fra.yaml' 2026-05-11T15:53:26,536 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_hau.yaml' 2026-05-11T15:53:26,537 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_ibo.yaml' 2026-05-11T15:53:26,538 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_kin.yaml' 2026-05-11T15:53:26,540 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lin.yaml' 2026-05-11T15:53:26,541 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_lug.yaml' 2026-05-11T15:53:26,542 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_orm.yaml' 2026-05-11T15:53:26,543 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sna.yaml' 2026-05-11T15:53:26,544 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_sot.yaml' 2026-05-11T15:53:26,546 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_swa.yaml' 2026-05-11T15:53:26,547 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_twi.yaml' 2026-05-11T15:53:26,548 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_wol.yaml' 2026-05-11T15:53:26,549 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_xho.yaml' 2026-05-11T15:53:26,550 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yaml' 2026-05-11T15:53:26,552 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_yor.yaml' 2026-05-11T15:53:26,553 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/afrixnli_translate_zul.yaml' 2026-05-11T15:53:26,554 adding 'lm_eval/tasks/afrixnli/translate/prompt_1/utils.py' 2026-05-11T15:53:26,556 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_amh.yaml' 2026-05-11T15:53:26,557 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ewe.yaml' 2026-05-11T15:53:26,558 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_fra.yaml' 2026-05-11T15:53:26,559 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_hau.yaml' 2026-05-11T15:53:26,561 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_ibo.yaml' 2026-05-11T15:53:26,562 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_kin.yaml' 2026-05-11T15:53:26,563 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lin.yaml' 2026-05-11T15:53:26,564 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_lug.yaml' 2026-05-11T15:53:26,565 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_orm.yaml' 2026-05-11T15:53:26,566 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sna.yaml' 2026-05-11T15:53:26,567 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_sot.yaml' 2026-05-11T15:53:26,569 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_swa.yaml' 2026-05-11T15:53:26,570 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_twi.yaml' 2026-05-11T15:53:26,571 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_wol.yaml' 2026-05-11T15:53:26,572 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_xho.yaml' 2026-05-11T15:53:26,574 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yaml' 2026-05-11T15:53:26,575 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_yor.yaml' 2026-05-11T15:53:26,576 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/afrixnli_translate_zul.yaml' 2026-05-11T15:53:26,577 adding 'lm_eval/tasks/afrixnli/translate/prompt_2/utils.py' 2026-05-11T15:53:26,579 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_amh.yaml' 2026-05-11T15:53:26,580 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ewe.yaml' 2026-05-11T15:53:26,581 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_fra.yaml' 2026-05-11T15:53:26,582 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_hau.yaml' 2026-05-11T15:53:26,583 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_ibo.yaml' 2026-05-11T15:53:26,585 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_kin.yaml' 2026-05-11T15:53:26,586 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lin.yaml' 2026-05-11T15:53:26,587 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_lug.yaml' 2026-05-11T15:53:26,588 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_orm.yaml' 2026-05-11T15:53:26,589 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sna.yaml' 2026-05-11T15:53:26,591 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_sot.yaml' 2026-05-11T15:53:26,592 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_swa.yaml' 2026-05-11T15:53:26,593 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_twi.yaml' 2026-05-11T15:53:26,594 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_wol.yaml' 2026-05-11T15:53:26,596 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_xho.yaml' 2026-05-11T15:53:26,597 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yaml' 2026-05-11T15:53:26,598 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_yor.yaml' 2026-05-11T15:53:26,599 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/afrixnli_translate_zul.yaml' 2026-05-11T15:53:26,600 adding 'lm_eval/tasks/afrixnli/translate/prompt_3/utils.py' 2026-05-11T15:53:26,603 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_amh.yaml' 2026-05-11T15:53:26,604 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ewe.yaml' 2026-05-11T15:53:26,605 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_fra.yaml' 2026-05-11T15:53:26,606 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_hau.yaml' 2026-05-11T15:53:26,607 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_ibo.yaml' 2026-05-11T15:53:26,609 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_kin.yaml' 2026-05-11T15:53:26,610 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lin.yaml' 2026-05-11T15:53:26,611 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_lug.yaml' 2026-05-11T15:53:26,612 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_orm.yaml' 2026-05-11T15:53:26,614 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sna.yaml' 2026-05-11T15:53:26,615 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_sot.yaml' 2026-05-11T15:53:26,616 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_swa.yaml' 2026-05-11T15:53:26,617 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_twi.yaml' 2026-05-11T15:53:26,619 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_wol.yaml' 2026-05-11T15:53:26,620 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_xho.yaml' 2026-05-11T15:53:26,621 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yaml' 2026-05-11T15:53:26,622 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_yor.yaml' 2026-05-11T15:53:26,623 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/afrixnli_translate_zul.yaml' 2026-05-11T15:53:26,625 adding 'lm_eval/tasks/afrixnli/translate/prompt_4/utils.py' 2026-05-11T15:53:26,626 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_amh.yaml' 2026-05-11T15:53:26,627 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ewe.yaml' 2026-05-11T15:53:26,629 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_fra.yaml' 2026-05-11T15:53:26,630 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_hau.yaml' 2026-05-11T15:53:26,631 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_ibo.yaml' 2026-05-11T15:53:26,632 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_kin.yaml' 2026-05-11T15:53:26,633 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lin.yaml' 2026-05-11T15:53:26,634 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_lug.yaml' 2026-05-11T15:53:26,635 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_orm.yaml' 2026-05-11T15:53:26,636 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sna.yaml' 2026-05-11T15:53:26,638 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_sot.yaml' 2026-05-11T15:53:26,639 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_swa.yaml' 2026-05-11T15:53:26,640 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_twi.yaml' 2026-05-11T15:53:26,641 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_wol.yaml' 2026-05-11T15:53:26,643 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_xho.yaml' 2026-05-11T15:53:26,644 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yaml' 2026-05-11T15:53:26,645 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_yor.yaml' 2026-05-11T15:53:26,646 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/afrixnli_translate_zul.yaml' 2026-05-11T15:53:26,647 adding 'lm_eval/tasks/afrixnli/translate/prompt_5/utils.py' 2026-05-11T15:53:26,650 adding 'lm_eval/tasks/afrobench/README.md' 2026-05-11T15:53:26,651 adding 'lm_eval/tasks/afrobench/afrobench-lite.yaml' 2026-05-11T15:53:26,652 adding 'lm_eval/tasks/afrobench/afrobench.yaml' 2026-05-11T15:53:26,654 adding 'lm_eval/tasks/afrobench/adr/README.md' 2026-05-11T15:53:26,655 adding 'lm_eval/tasks/afrobench/adr/afridiacritics.yaml' 2026-05-11T15:53:26,657 adding 'lm_eval/tasks/afrobench/adr/gen_utils.py' 2026-05-11T15:53:26,658 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_bbj.yaml' 2026-05-11T15:53:26,660 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_fon.yaml' 2026-05-11T15:53:26,661 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_ibo.yaml' 2026-05-11T15:53:26,662 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_wol.yaml' 2026-05-11T15:53:26,663 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yaml' 2026-05-11T15:53:26,665 adding 'lm_eval/tasks/afrobench/adr/prompt_1/afridiacritics_yor.yaml' 2026-05-11T15:53:26,666 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_bbj.yaml' 2026-05-11T15:53:26,667 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_fon.yaml' 2026-05-11T15:53:26,669 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_ibo.yaml' 2026-05-11T15:53:26,670 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_wol.yaml' 2026-05-11T15:53:26,671 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yaml' 2026-05-11T15:53:26,672 adding 'lm_eval/tasks/afrobench/adr/prompt_2/afridiacritics_yor.yaml' 2026-05-11T15:53:26,674 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_bbj.yaml' 2026-05-11T15:53:26,675 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_fon.yaml' 2026-05-11T15:53:26,677 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_ibo.yaml' 2026-05-11T15:53:26,678 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_wol.yaml' 2026-05-11T15:53:26,679 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yaml' 2026-05-11T15:53:26,680 adding 'lm_eval/tasks/afrobench/adr/prompt_3/afridiacritics_yor.yaml' 2026-05-11T15:53:26,682 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_bbj.yaml' 2026-05-11T15:53:26,683 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_fon.yaml' 2026-05-11T15:53:26,684 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_ibo.yaml' 2026-05-11T15:53:26,685 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_wol.yaml' 2026-05-11T15:53:26,686 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yaml' 2026-05-11T15:53:26,687 adding 'lm_eval/tasks/afrobench/adr/prompt_4/afridiacritics_yor.yaml' 2026-05-11T15:53:26,689 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_bbj.yaml' 2026-05-11T15:53:26,690 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_fon.yaml' 2026-05-11T15:53:26,691 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_ibo.yaml' 2026-05-11T15:53:26,693 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_wol.yaml' 2026-05-11T15:53:26,694 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yaml' 2026-05-11T15:53:26,695 adding 'lm_eval/tasks/afrobench/adr/prompt_5/afridiacritics_yor.yaml' 2026-05-11T15:53:26,697 adding 'lm_eval/tasks/afrobench/afriqa/README.md' 2026-05-11T15:53:26,698 adding 'lm_eval/tasks/afrobench/afriqa/afriqa.yaml' 2026-05-11T15:53:26,700 adding 'lm_eval/tasks/afrobench/afriqa/utils.py' 2026-05-11T15:53:26,702 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa' 2026-05-11T15:53:26,703 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_bem.yaml' 2026-05-11T15:53:26,704 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_fon.yaml' 2026-05-11T15:53:26,705 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_hau.yaml' 2026-05-11T15:53:26,707 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_ibo.yaml' 2026-05-11T15:53:26,708 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_kin.yaml' 2026-05-11T15:53:26,709 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_swa.yaml' 2026-05-11T15:53:26,710 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_twi.yaml' 2026-05-11T15:53:26,711 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_yor.yaml' 2026-05-11T15:53:26,713 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/afriqa_zul.yaml' 2026-05-11T15:53:26,714 adding 'lm_eval/tasks/afrobench/afriqa/prompt_1/utils.py' 2026-05-11T15:53:26,716 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa' 2026-05-11T15:53:26,717 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_bem.yaml' 2026-05-11T15:53:26,718 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_fon.yaml' 2026-05-11T15:53:26,719 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_hau.yaml' 2026-05-11T15:53:26,720 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_ibo.yaml' 2026-05-11T15:53:26,722 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_kin.yaml' 2026-05-11T15:53:26,723 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_swa.yaml' 2026-05-11T15:53:26,724 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_twi.yaml' 2026-05-11T15:53:26,725 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_yor.yaml' 2026-05-11T15:53:26,726 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/afriqa_zul.yaml' 2026-05-11T15:53:26,728 adding 'lm_eval/tasks/afrobench/afriqa/prompt_2/utils.py' 2026-05-11T15:53:26,730 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa' 2026-05-11T15:53:26,731 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_bem.yaml' 2026-05-11T15:53:26,733 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_fon.yaml' 2026-05-11T15:53:26,734 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_hau.yaml' 2026-05-11T15:53:26,735 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_ibo.yaml' 2026-05-11T15:53:26,736 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_kin.yaml' 2026-05-11T15:53:26,737 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_swa.yaml' 2026-05-11T15:53:26,739 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_twi.yaml' 2026-05-11T15:53:26,740 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_yor.yaml' 2026-05-11T15:53:26,741 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/afriqa_zul.yaml' 2026-05-11T15:53:26,743 adding 'lm_eval/tasks/afrobench/afriqa/prompt_3/utils.py' 2026-05-11T15:53:26,744 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa' 2026-05-11T15:53:26,746 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_bem.yaml' 2026-05-11T15:53:26,747 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_fon.yaml' 2026-05-11T15:53:26,748 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_hau.yaml' 2026-05-11T15:53:26,749 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_ibo.yaml' 2026-05-11T15:53:26,750 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_kin.yaml' 2026-05-11T15:53:26,752 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_swa.yaml' 2026-05-11T15:53:26,753 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_twi.yaml' 2026-05-11T15:53:26,754 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_yor.yaml' 2026-05-11T15:53:26,756 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/afriqa_zul.yaml' 2026-05-11T15:53:26,757 adding 'lm_eval/tasks/afrobench/afriqa/prompt_4/utils.py' 2026-05-11T15:53:26,759 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa' 2026-05-11T15:53:26,760 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_bem.yaml' 2026-05-11T15:53:26,761 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_fon.yaml' 2026-05-11T15:53:26,762 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_hau.yaml' 2026-05-11T15:53:26,763 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_ibo.yaml' 2026-05-11T15:53:26,765 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_kin.yaml' 2026-05-11T15:53:26,766 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_swa.yaml' 2026-05-11T15:53:26,767 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_twi.yaml' 2026-05-11T15:53:26,768 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_yor.yaml' 2026-05-11T15:53:26,770 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/afriqa_zul.yaml' 2026-05-11T15:53:26,771 adding 'lm_eval/tasks/afrobench/afriqa/prompt_5/utils.py' 2026-05-11T15:53:26,773 adding 'lm_eval/tasks/afrobench/afrisenti/README.md' 2026-05-11T15:53:26,775 adding 'lm_eval/tasks/afrobench/afrisenti/afrisenti.yaml' 2026-05-11T15:53:26,776 adding 'lm_eval/tasks/afrobench/afrisenti/fewshot.sh' 2026-05-11T15:53:26,777 adding 'lm_eval/tasks/afrobench/afrisenti/utils.py' 2026-05-11T15:53:26,780 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti' 2026-05-11T15:53:26,781 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_amh.yaml' 2026-05-11T15:53:26,782 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_arq.yaml' 2026-05-11T15:53:26,783 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ary.yaml' 2026-05-11T15:53:26,784 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_hau.yaml' 2026-05-11T15:53:26,785 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_ibo.yaml' 2026-05-11T15:53:26,787 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_kin.yaml' 2026-05-11T15:53:26,788 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_orm.yaml' 2026-05-11T15:53:26,789 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_pcm.yaml' 2026-05-11T15:53:26,790 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_por.yaml' 2026-05-11T15:53:26,791 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_swa.yaml' 2026-05-11T15:53:26,793 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tir.yaml' 2026-05-11T15:53:26,794 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_tso.yaml' 2026-05-11T15:53:26,795 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_twi.yaml' 2026-05-11T15:53:26,796 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/afrisenti_yor.yaml' 2026-05-11T15:53:26,797 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/run.sh' 2026-05-11T15:53:26,798 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/utils.py' 2026-05-11T15:53:26,800 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_1/xx.py' 2026-05-11T15:53:26,801 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti' 2026-05-11T15:53:26,803 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_amh.yaml' 2026-05-11T15:53:26,804 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_arq.yaml' 2026-05-11T15:53:26,805 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ary.yaml' 2026-05-11T15:53:26,806 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_hau.yaml' 2026-05-11T15:53:26,807 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_ibo.yaml' 2026-05-11T15:53:26,808 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_kin.yaml' 2026-05-11T15:53:26,810 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_orm.yaml' 2026-05-11T15:53:26,811 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_pcm.yaml' 2026-05-11T15:53:26,812 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_por.yaml' 2026-05-11T15:53:26,813 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_swa.yaml' 2026-05-11T15:53:26,814 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tir.yaml' 2026-05-11T15:53:26,815 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_tso.yaml' 2026-05-11T15:53:26,817 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_twi.yaml' 2026-05-11T15:53:26,818 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/afrisenti_yor.yaml' 2026-05-11T15:53:26,819 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/run.sh' 2026-05-11T15:53:26,820 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/utils.py' 2026-05-11T15:53:26,822 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_2/xx.py' 2026-05-11T15:53:26,824 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti' 2026-05-11T15:53:26,825 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_amh.yaml' 2026-05-11T15:53:26,852 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_arq.yaml' 2026-05-11T15:53:26,854 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ary.yaml' 2026-05-11T15:53:26,855 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_hau.yaml' 2026-05-11T15:53:26,856 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_ibo.yaml' 2026-05-11T15:53:26,858 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_kin.yaml' 2026-05-11T15:53:26,859 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_orm.yaml' 2026-05-11T15:53:26,860 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_pcm.yaml' 2026-05-11T15:53:26,861 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_por.yaml' 2026-05-11T15:53:26,862 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_swa.yaml' 2026-05-11T15:53:26,864 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tir.yaml' 2026-05-11T15:53:26,865 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_tso.yaml' 2026-05-11T15:53:26,866 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_twi.yaml' 2026-05-11T15:53:26,867 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/afrisenti_yor.yaml' 2026-05-11T15:53:26,869 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/utils.py' 2026-05-11T15:53:26,870 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_3/xx.py' 2026-05-11T15:53:26,872 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti' 2026-05-11T15:53:26,873 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_amh.yaml' 2026-05-11T15:53:26,874 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_arq.yaml' 2026-05-11T15:53:26,875 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ary.yaml' 2026-05-11T15:53:26,877 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_hau.yaml' 2026-05-11T15:53:26,878 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_ibo.yaml' 2026-05-11T15:53:26,879 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_kin.yaml' 2026-05-11T15:53:26,880 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_orm.yaml' 2026-05-11T15:53:26,881 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_pcm.yaml' 2026-05-11T15:53:26,882 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_por.yaml' 2026-05-11T15:53:26,883 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_swa.yaml' 2026-05-11T15:53:26,885 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tir.yaml' 2026-05-11T15:53:26,886 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_tso.yaml' 2026-05-11T15:53:26,887 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_twi.yaml' 2026-05-11T15:53:26,888 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/afrisenti_yor.yaml' 2026-05-11T15:53:26,889 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/utils.py' 2026-05-11T15:53:26,890 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_4/xx.py' 2026-05-11T15:53:26,892 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti' 2026-05-11T15:53:26,893 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_amh.yaml' 2026-05-11T15:53:26,894 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_arq.yaml' 2026-05-11T15:53:26,896 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ary.yaml' 2026-05-11T15:53:26,897 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_hau.yaml' 2026-05-11T15:53:26,898 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_ibo.yaml' 2026-05-11T15:53:26,900 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_kin.yaml' 2026-05-11T15:53:26,901 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_orm.yaml' 2026-05-11T15:53:26,902 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_pcm.yaml' 2026-05-11T15:53:26,903 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_por.yaml' 2026-05-11T15:53:26,904 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_swa.yaml' 2026-05-11T15:53:26,906 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tir.yaml' 2026-05-11T15:53:26,907 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_tso.yaml' 2026-05-11T15:53:26,908 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_twi.yaml' 2026-05-11T15:53:26,910 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/afrisenti_yor.yaml' 2026-05-11T15:53:26,911 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/utils.py' 2026-05-11T15:53:26,912 adding 'lm_eval/tasks/afrobench/afrisenti/prompt_5/xx.py' 2026-05-11T15:53:26,914 adding 'lm_eval/tasks/afrobench/belebele/README.md' 2026-05-11T15:53:26,915 adding 'lm_eval/tasks/afrobench/belebele/belebele.yaml' 2026-05-11T15:53:26,917 adding 'lm_eval/tasks/afrobench/belebele/utils.py' 2026-05-11T15:53:26,919 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele' 2026-05-11T15:53:26,920 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_afr.yaml' 2026-05-11T15:53:26,921 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_amh.yaml' 2026-05-11T15:53:26,922 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ary.yaml' 2026-05-11T15:53:26,924 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_arz.yaml' 2026-05-11T15:53:26,925 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_bam.yaml' 2026-05-11T15:53:26,926 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_eng.yaml' 2026-05-11T15:53:26,927 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fra.yaml' 2026-05-11T15:53:26,928 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_fuv.yaml' 2026-05-11T15:53:26,930 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_gaz.yaml' 2026-05-11T15:53:26,931 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_hau.yaml' 2026-05-11T15:53:26,932 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ibo.yaml' 2026-05-11T15:53:26,933 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kea.yaml' 2026-05-11T15:53:26,934 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_kin.yaml' 2026-05-11T15:53:26,935 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lin.yaml' 2026-05-11T15:53:26,936 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_lug.yaml' 2026-05-11T15:53:26,938 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_luo.yaml' 2026-05-11T15:53:26,939 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_nya.yaml' 2026-05-11T15:53:26,940 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_plt.yaml' 2026-05-11T15:53:26,941 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_por.yaml' 2026-05-11T15:53:26,942 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sna.yaml' 2026-05-11T15:53:26,943 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_som.yaml' 2026-05-11T15:53:26,945 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_sot.yaml' 2026-05-11T15:53:26,946 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_ssw.yaml' 2026-05-11T15:53:26,947 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_swa.yaml' 2026-05-11T15:53:26,948 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tir.yaml' 2026-05-11T15:53:26,949 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tsn.yaml' 2026-05-11T15:53:26,950 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_tso.yaml' 2026-05-11T15:53:26,952 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_wol.yaml' 2026-05-11T15:53:26,953 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_xho.yaml' 2026-05-11T15:53:26,954 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_yor.yaml' 2026-05-11T15:53:26,955 adding 'lm_eval/tasks/afrobench/belebele/prompt_1/belebele_zul.yaml' 2026-05-11T15:53:26,958 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele' 2026-05-11T15:53:26,959 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_afr.yaml' 2026-05-11T15:53:26,960 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_amh.yaml' 2026-05-11T15:53:26,962 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ary.yaml' 2026-05-11T15:53:26,963 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_arz.yaml' 2026-05-11T15:53:26,964 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_bam.yaml' 2026-05-11T15:53:26,965 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_eng.yaml' 2026-05-11T15:53:26,966 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fra.yaml' 2026-05-11T15:53:26,967 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_fuv.yaml' 2026-05-11T15:53:26,969 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_gaz.yaml' 2026-05-11T15:53:26,970 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_hau.yaml' 2026-05-11T15:53:26,971 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ibo.yaml' 2026-05-11T15:53:26,972 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kea.yaml' 2026-05-11T15:53:26,973 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_kin.yaml' 2026-05-11T15:53:26,975 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lin.yaml' 2026-05-11T15:53:26,976 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_lug.yaml' 2026-05-11T15:53:26,977 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_luo.yaml' 2026-05-11T15:53:26,978 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_nya.yaml' 2026-05-11T15:53:26,979 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_plt.yaml' 2026-05-11T15:53:26,981 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_por.yaml' 2026-05-11T15:53:26,982 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sna.yaml' 2026-05-11T15:53:26,983 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_som.yaml' 2026-05-11T15:53:26,984 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_sot.yaml' 2026-05-11T15:53:26,985 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_ssw.yaml' 2026-05-11T15:53:26,987 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_swa.yaml' 2026-05-11T15:53:26,988 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tir.yaml' 2026-05-11T15:53:26,989 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tsn.yaml' 2026-05-11T15:53:26,990 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_tso.yaml' 2026-05-11T15:53:26,991 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_wol.yaml' 2026-05-11T15:53:26,992 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_xho.yaml' 2026-05-11T15:53:26,994 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_yor.yaml' 2026-05-11T15:53:26,995 adding 'lm_eval/tasks/afrobench/belebele/prompt_2/belebele_zul.yaml' 2026-05-11T15:53:26,997 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele' 2026-05-11T15:53:26,998 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_afr.yaml' 2026-05-11T15:53:27,000 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_amh.yaml' 2026-05-11T15:53:27,001 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ary.yaml' 2026-05-11T15:53:27,002 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_arz.yaml' 2026-05-11T15:53:27,003 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_bam.yaml' 2026-05-11T15:53:27,005 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_eng.yaml' 2026-05-11T15:53:27,006 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fra.yaml' 2026-05-11T15:53:27,007 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_fuv.yaml' 2026-05-11T15:53:27,008 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_gaz.yaml' 2026-05-11T15:53:27,010 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_hau.yaml' 2026-05-11T15:53:27,011 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ibo.yaml' 2026-05-11T15:53:27,012 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kea.yaml' 2026-05-11T15:53:27,013 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_kin.yaml' 2026-05-11T15:53:27,015 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lin.yaml' 2026-05-11T15:53:27,016 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_lug.yaml' 2026-05-11T15:53:27,017 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_luo.yaml' 2026-05-11T15:53:27,018 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_nya.yaml' 2026-05-11T15:53:27,019 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_plt.yaml' 2026-05-11T15:53:27,021 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_por.yaml' 2026-05-11T15:53:27,022 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sna.yaml' 2026-05-11T15:53:27,023 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_som.yaml' 2026-05-11T15:53:27,024 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_sot.yaml' 2026-05-11T15:53:27,026 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_ssw.yaml' 2026-05-11T15:53:27,027 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_swa.yaml' 2026-05-11T15:53:27,028 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tir.yaml' 2026-05-11T15:53:27,030 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tsn.yaml' 2026-05-11T15:53:27,031 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_tso.yaml' 2026-05-11T15:53:27,032 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_wol.yaml' 2026-05-11T15:53:27,033 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_xho.yaml' 2026-05-11T15:53:27,034 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_yor.yaml' 2026-05-11T15:53:27,036 adding 'lm_eval/tasks/afrobench/belebele/prompt_3/belebele_zul.yaml' 2026-05-11T15:53:27,038 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele' 2026-05-11T15:53:27,039 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_afr.yaml' 2026-05-11T15:53:27,040 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_amh.yaml' 2026-05-11T15:53:27,042 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ary.yaml' 2026-05-11T15:53:27,043 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_arz.yaml' 2026-05-11T15:53:27,044 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_bam.yaml' 2026-05-11T15:53:27,045 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_eng.yaml' 2026-05-11T15:53:27,046 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fra.yaml' 2026-05-11T15:53:27,048 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_fuv.yaml' 2026-05-11T15:53:27,049 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_gaz.yaml' 2026-05-11T15:53:27,050 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_hau.yaml' 2026-05-11T15:53:27,051 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ibo.yaml' 2026-05-11T15:53:27,053 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kea.yaml' 2026-05-11T15:53:27,054 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_kin.yaml' 2026-05-11T15:53:27,055 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lin.yaml' 2026-05-11T15:53:27,056 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_lug.yaml' 2026-05-11T15:53:27,057 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_luo.yaml' 2026-05-11T15:53:27,059 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_nya.yaml' 2026-05-11T15:53:27,060 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_plt.yaml' 2026-05-11T15:53:27,061 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_por.yaml' 2026-05-11T15:53:27,062 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sna.yaml' 2026-05-11T15:53:27,064 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_som.yaml' 2026-05-11T15:53:27,065 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_sot.yaml' 2026-05-11T15:53:27,066 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_ssw.yaml' 2026-05-11T15:53:27,068 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_swa.yaml' 2026-05-11T15:53:27,069 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tir.yaml' 2026-05-11T15:53:27,070 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tsn.yaml' 2026-05-11T15:53:27,071 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_tso.yaml' 2026-05-11T15:53:27,073 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_wol.yaml' 2026-05-11T15:53:27,074 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_xho.yaml' 2026-05-11T15:53:27,075 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_yor.yaml' 2026-05-11T15:53:27,076 adding 'lm_eval/tasks/afrobench/belebele/prompt_4/belebele_zul.yaml' 2026-05-11T15:53:27,079 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele' 2026-05-11T15:53:27,080 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_afr.yaml' 2026-05-11T15:53:27,081 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_amh.yaml' 2026-05-11T15:53:27,082 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ary.yaml' 2026-05-11T15:53:27,084 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_arz.yaml' 2026-05-11T15:53:27,085 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_bam.yaml' 2026-05-11T15:53:27,086 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_eng.yaml' 2026-05-11T15:53:27,087 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fra.yaml' 2026-05-11T15:53:27,088 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_fuv.yaml' 2026-05-11T15:53:27,090 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_gaz.yaml' 2026-05-11T15:53:27,091 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_hau.yaml' 2026-05-11T15:53:27,092 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ibo.yaml' 2026-05-11T15:53:27,094 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kea.yaml' 2026-05-11T15:53:27,095 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_kin.yaml' 2026-05-11T15:53:27,096 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lin.yaml' 2026-05-11T15:53:27,097 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_lug.yaml' 2026-05-11T15:53:27,098 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_luo.yaml' 2026-05-11T15:53:27,099 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_nya.yaml' 2026-05-11T15:53:27,100 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_plt.yaml' 2026-05-11T15:53:27,102 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_por.yaml' 2026-05-11T15:53:27,103 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sna.yaml' 2026-05-11T15:53:27,104 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_som.yaml' 2026-05-11T15:53:27,105 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_sot.yaml' 2026-05-11T15:53:27,106 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_ssw.yaml' 2026-05-11T15:53:27,107 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_swa.yaml' 2026-05-11T15:53:27,109 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tir.yaml' 2026-05-11T15:53:27,110 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tsn.yaml' 2026-05-11T15:53:27,111 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_tso.yaml' 2026-05-11T15:53:27,112 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_wol.yaml' 2026-05-11T15:53:27,113 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_xho.yaml' 2026-05-11T15:53:27,115 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_yor.yaml' 2026-05-11T15:53:27,116 adding 'lm_eval/tasks/afrobench/belebele/prompt_5/belebele_zul.yaml' 2026-05-11T15:53:27,118 adding 'lm_eval/tasks/afrobench/flores/README.md' 2026-05-11T15:53:27,119 adding 'lm_eval/tasks/afrobench/flores/flores.yaml' 2026-05-11T15:53:27,121 adding 'lm_eval/tasks/afrobench/flores/gen_utils.py' 2026-05-11T15:53:27,123 adding 'lm_eval/tasks/afrobench/flores/prompt_1/flores' 2026-05-11T15:53:27,126 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores' 2026-05-11T15:53:27,127 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,128 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ace_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,130 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_acq_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,131 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aeb_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,132 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_afr_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,133 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_aka_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,134 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_amh_Ethi-eng_Latn.yaml' 2026-05-11T15:53:27,136 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ary_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,137 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_arz_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,139 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bam_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,140 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ban_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,141 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_bem_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,142 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_cjk_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,143 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dik_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,145 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_dyu_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,146 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ewe_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,147 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fon_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,148 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fra_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,149 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_fuv_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,150 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_gaz_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,152 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_hau_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,153 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ibo_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,154 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kab_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,155 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kam_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,157 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kbp_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,158 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kea_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,159 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kik_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,160 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kin_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,161 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kmb_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,163 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,164 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_knc_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,165 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_kon_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,166 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lin_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,168 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lua_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,169 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_lug_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,170 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_luo_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,171 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_mos_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,173 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nso_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,174 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nus_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,175 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_nya_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,176 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_plt_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,178 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_run_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,179 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sag_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,180 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sna_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,182 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_som_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,183 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sot_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,184 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_ssw_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,185 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_sun_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,186 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_swh_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,188 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,189 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_taq_Tfng-eng_Latn.yaml' 2026-05-11T15:53:27,190 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tir_Ethi-eng_Latn.yaml' 2026-05-11T15:53:27,191 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tsn_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,192 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tso_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,194 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tum_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,195 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_twi_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,196 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_tzm_Tfng-eng_Latn.yaml' 2026-05-11T15:53:27,197 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_umb_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,199 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_wol_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,200 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_xho_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,201 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_yor_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,202 adding 'lm_eval/tasks/afrobench/flores/prompt_1/african-english/flores_zul_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,205 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores' 2026-05-11T15:53:27,206 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Arab.yaml' 2026-05-11T15:53:27,207 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ace_Latn.yaml' 2026-05-11T15:53:27,209 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-acq_Arab.yaml' 2026-05-11T15:53:27,210 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aeb_Arab.yaml' 2026-05-11T15:53:27,211 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-afr_Latn.yaml' 2026-05-11T15:53:27,212 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-aka_Latn.yaml' 2026-05-11T15:53:27,213 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-amh_Ethi.yaml' 2026-05-11T15:53:27,214 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ary_Arab.yaml' 2026-05-11T15:53:27,216 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-arz_Arab.yaml' 2026-05-11T15:53:27,217 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bam_Latn.yaml' 2026-05-11T15:53:27,218 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ban_Latn.yaml' 2026-05-11T15:53:27,220 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-bem_Latn.yaml' 2026-05-11T15:53:27,221 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-cjk_Latn.yaml' 2026-05-11T15:53:27,222 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dik_Latn.yaml' 2026-05-11T15:53:27,223 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-dyu_Latn.yaml' 2026-05-11T15:53:27,224 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ewe_Latn.yaml' 2026-05-11T15:53:27,226 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fon_Latn.yaml' 2026-05-11T15:53:27,227 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fra_Latn.yaml' 2026-05-11T15:53:27,228 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-fuv_Latn.yaml' 2026-05-11T15:53:27,229 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-gaz_Latn.yaml' 2026-05-11T15:53:27,231 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-hau_Latn.yaml' 2026-05-11T15:53:27,232 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ibo_Latn.yaml' 2026-05-11T15:53:27,233 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kab_Latn.yaml' 2026-05-11T15:53:27,234 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kam_Latn.yaml' 2026-05-11T15:53:27,235 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kbp_Latn.yaml' 2026-05-11T15:53:27,237 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kea_Latn.yaml' 2026-05-11T15:53:27,238 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kik_Latn.yaml' 2026-05-11T15:53:27,239 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kin_Latn.yaml' 2026-05-11T15:53:27,240 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kmb_Latn.yaml' 2026-05-11T15:53:27,242 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Arab.yaml' 2026-05-11T15:53:27,243 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-knc_Latn.yaml' 2026-05-11T15:53:27,244 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-kon_Latn.yaml' 2026-05-11T15:53:27,245 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lin_Latn.yaml' 2026-05-11T15:53:27,247 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lua_Latn.yaml' 2026-05-11T15:53:27,248 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-lug_Latn.yaml' 2026-05-11T15:53:27,249 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-luo_Latn.yaml' 2026-05-11T15:53:27,250 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-mos_Latn.yaml' 2026-05-11T15:53:27,252 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nso_Latn.yaml' 2026-05-11T15:53:27,253 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nus_Latn.yaml' 2026-05-11T15:53:27,254 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-nya_Latn.yaml' 2026-05-11T15:53:27,255 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-plt_Latn.yaml' 2026-05-11T15:53:27,256 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-run_Latn.yaml' 2026-05-11T15:53:27,258 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sag_Latn.yaml' 2026-05-11T15:53:27,259 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sna_Latn.yaml' 2026-05-11T15:53:27,260 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-som_Latn.yaml' 2026-05-11T15:53:27,261 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sot_Latn.yaml' 2026-05-11T15:53:27,262 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-ssw_Latn.yaml' 2026-05-11T15:53:27,264 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-sun_Latn.yaml' 2026-05-11T15:53:27,265 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-swh_Latn.yaml' 2026-05-11T15:53:27,266 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Latn.yaml' 2026-05-11T15:53:27,267 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-taq_Tfng.yaml' 2026-05-11T15:53:27,269 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tir_Ethi.yaml' 2026-05-11T15:53:27,270 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tsn_Latn.yaml' 2026-05-11T15:53:27,271 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tso_Latn.yaml' 2026-05-11T15:53:27,272 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tum_Latn.yaml' 2026-05-11T15:53:27,274 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-twi_Latn.yaml' 2026-05-11T15:53:27,275 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-tzm_Tfng.yaml' 2026-05-11T15:53:27,276 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-umb_Latn.yaml' 2026-05-11T15:53:27,277 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-wol_Latn.yaml' 2026-05-11T15:53:27,278 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-xho_Latn.yaml' 2026-05-11T15:53:27,279 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-yor_Latn.yaml' 2026-05-11T15:53:27,281 adding 'lm_eval/tasks/afrobench/flores/prompt_1/english-african/flores_eng_Latn-zul_Latn.yaml' 2026-05-11T15:53:27,282 adding 'lm_eval/tasks/afrobench/flores/prompt_2/flores' 2026-05-11T15:53:27,285 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores' 2026-05-11T15:53:27,286 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,287 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ace_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,288 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_acq_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,290 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aeb_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,291 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_afr_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,292 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_aka_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,293 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_amh_Ethi-eng_Latn.yaml' 2026-05-11T15:53:27,294 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ary_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,295 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_arz_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,297 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bam_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,298 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ban_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,299 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_bem_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,300 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_cjk_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,302 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dik_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,303 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_dyu_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,304 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ewe_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,305 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fon_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,307 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fra_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,308 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_fuv_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,309 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_gaz_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,310 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_hau_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,312 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ibo_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,313 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kab_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,314 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kam_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,315 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kbp_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,316 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kea_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,318 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kik_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,319 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kin_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,320 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kmb_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,321 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,322 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_knc_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,323 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_kon_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,325 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lin_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,326 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lua_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,327 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_lug_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,328 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_luo_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,330 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_mos_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,331 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nso_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,332 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nus_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,333 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_nya_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,335 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_plt_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,336 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_run_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,337 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sag_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,339 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sna_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,340 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_som_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,341 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sot_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,342 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_ssw_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,343 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_sun_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,345 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_swh_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,346 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,347 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_taq_Tfng-eng_Latn.yaml' 2026-05-11T15:53:27,349 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tir_Ethi-eng_Latn.yaml' 2026-05-11T15:53:27,350 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tsn_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,351 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tso_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,352 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tum_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,353 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_twi_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,355 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_tzm_Tfng-eng_Latn.yaml' 2026-05-11T15:53:27,356 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_umb_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,357 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_wol_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,359 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_xho_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,360 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_yor_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,361 adding 'lm_eval/tasks/afrobench/flores/prompt_2/african-english/flores_zul_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,364 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores' 2026-05-11T15:53:27,365 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Arab.yaml' 2026-05-11T15:53:27,367 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ace_Latn.yaml' 2026-05-11T15:53:27,368 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-acq_Arab.yaml' 2026-05-11T15:53:27,369 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aeb_Arab.yaml' 2026-05-11T15:53:27,370 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-afr_Latn.yaml' 2026-05-11T15:53:27,371 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-aka_Latn.yaml' 2026-05-11T15:53:27,373 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-amh_Ethi.yaml' 2026-05-11T15:53:27,374 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ary_Arab.yaml' 2026-05-11T15:53:27,375 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-arz_Arab.yaml' 2026-05-11T15:53:27,376 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bam_Latn.yaml' 2026-05-11T15:53:27,377 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ban_Latn.yaml' 2026-05-11T15:53:27,379 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-bem_Latn.yaml' 2026-05-11T15:53:27,380 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-cjk_Latn.yaml' 2026-05-11T15:53:27,381 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dik_Latn.yaml' 2026-05-11T15:53:27,382 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-dyu_Latn.yaml' 2026-05-11T15:53:27,384 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ewe_Latn.yaml' 2026-05-11T15:53:27,385 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fon_Latn.yaml' 2026-05-11T15:53:27,386 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fra_Latn.yaml' 2026-05-11T15:53:27,387 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-fuv_Latn.yaml' 2026-05-11T15:53:27,388 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-gaz_Latn.yaml' 2026-05-11T15:53:27,390 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-hau_Latn.yaml' 2026-05-11T15:53:27,391 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ibo_Latn.yaml' 2026-05-11T15:53:27,392 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kab_Latn.yaml' 2026-05-11T15:53:27,393 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kam_Latn.yaml' 2026-05-11T15:53:27,394 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kbp_Latn.yaml' 2026-05-11T15:53:27,396 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kea_Latn.yaml' 2026-05-11T15:53:27,397 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kik_Latn.yaml' 2026-05-11T15:53:27,398 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kin_Latn.yaml' 2026-05-11T15:53:27,399 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kmb_Latn.yaml' 2026-05-11T15:53:27,401 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Arab.yaml' 2026-05-11T15:53:27,402 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-knc_Latn.yaml' 2026-05-11T15:53:27,403 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-kon_Latn.yaml' 2026-05-11T15:53:27,404 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lin_Latn.yaml' 2026-05-11T15:53:27,405 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lua_Latn.yaml' 2026-05-11T15:53:27,407 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-lug_Latn.yaml' 2026-05-11T15:53:27,408 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-luo_Latn.yaml' 2026-05-11T15:53:27,409 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-mos_Latn.yaml' 2026-05-11T15:53:27,410 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nso_Latn.yaml' 2026-05-11T15:53:27,411 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nus_Latn.yaml' 2026-05-11T15:53:27,413 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-nya_Latn.yaml' 2026-05-11T15:53:27,414 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-plt_Latn.yaml' 2026-05-11T15:53:27,415 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-run_Latn.yaml' 2026-05-11T15:53:27,416 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sag_Latn.yaml' 2026-05-11T15:53:27,417 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sna_Latn.yaml' 2026-05-11T15:53:27,419 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-som_Latn.yaml' 2026-05-11T15:53:27,420 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sot_Latn.yaml' 2026-05-11T15:53:27,421 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-ssw_Latn.yaml' 2026-05-11T15:53:27,422 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-sun_Latn.yaml' 2026-05-11T15:53:27,424 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-swh_Latn.yaml' 2026-05-11T15:53:27,425 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Latn.yaml' 2026-05-11T15:53:27,426 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-taq_Tfng.yaml' 2026-05-11T15:53:27,428 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tir_Ethi.yaml' 2026-05-11T15:53:27,429 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tsn_Latn.yaml' 2026-05-11T15:53:27,430 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tso_Latn.yaml' 2026-05-11T15:53:27,431 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tum_Latn.yaml' 2026-05-11T15:53:27,433 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-twi_Latn.yaml' 2026-05-11T15:53:27,434 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-tzm_Tfng.yaml' 2026-05-11T15:53:27,435 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-umb_Latn.yaml' 2026-05-11T15:53:27,436 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-wol_Latn.yaml' 2026-05-11T15:53:27,437 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-xho_Latn.yaml' 2026-05-11T15:53:27,439 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-yor_Latn.yaml' 2026-05-11T15:53:27,440 adding 'lm_eval/tasks/afrobench/flores/prompt_2/english-african/flores_eng_Latn-zul_Latn.yaml' 2026-05-11T15:53:27,442 adding 'lm_eval/tasks/afrobench/flores/prompt_3/flores' 2026-05-11T15:53:27,445 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores' 2026-05-11T15:53:27,446 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,447 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ace_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,448 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_acq_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,449 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aeb_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,450 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_afr_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,452 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_aka_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,453 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_amh_Ethi-eng_Latn.yaml' 2026-05-11T15:53:27,454 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ary_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,455 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_arz_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,457 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bam_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,458 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ban_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,459 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_bem_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,460 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_cjk_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,462 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dik_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,463 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_dyu_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,464 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ewe_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,466 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fon_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,467 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fra_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,468 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_fuv_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,469 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_gaz_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,470 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_hau_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,472 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ibo_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,473 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kab_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,474 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kam_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,475 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kbp_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,477 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kea_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,478 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kik_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,479 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kin_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,480 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kmb_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,481 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Arab-eng_Latn.yaml' 2026-05-11T15:53:27,483 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_knc_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,484 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_kon_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,485 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lin_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,486 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lua_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,487 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_lug_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,488 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_luo_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,490 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_mos_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,491 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nso_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,492 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nus_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,493 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_nya_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,494 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_plt_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,495 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_run_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,496 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sag_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,498 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sna_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,499 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_som_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,500 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sot_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,501 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_ssw_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,502 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_sun_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,503 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_swh_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,505 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,506 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_taq_Tfng-eng_Latn.yaml' 2026-05-11T15:53:27,507 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tir_Ethi-eng_Latn.yaml' 2026-05-11T15:53:27,508 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tsn_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,509 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tso_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,511 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tum_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,512 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_twi_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,513 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_tzm_Tfng-eng_Latn.yaml' 2026-05-11T15:53:27,514 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_umb_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,516 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_wol_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,517 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_xho_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,518 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_yor_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,519 adding 'lm_eval/tasks/afrobench/flores/prompt_3/african-english/flores_zul_Latn-eng_Latn.yaml' 2026-05-11T15:53:27,522 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores' 2026-05-11T15:53:27,523 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Arab.yaml' 2026-05-11T15:53:27,524 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ace_Latn.yaml' 2026-05-11T15:53:27,526 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-acq_Arab.yaml' 2026-05-11T15:53:27,527 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aeb_Arab.yaml' 2026-05-11T15:53:27,528 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-afr_Latn.yaml' 2026-05-11T15:53:27,529 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-aka_Latn.yaml' 2026-05-11T15:53:27,530 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-amh_Ethi.yaml' 2026-05-11T15:53:27,531 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ary_Arab.yaml' 2026-05-11T15:53:27,533 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-arz_Arab.yaml' 2026-05-11T15:53:27,534 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bam_Latn.yaml' 2026-05-11T15:53:27,535 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ban_Latn.yaml' 2026-05-11T15:53:27,536 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-bem_Latn.yaml' 2026-05-11T15:53:27,537 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-cjk_Latn.yaml' 2026-05-11T15:53:27,538 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dik_Latn.yaml' 2026-05-11T15:53:27,540 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-dyu_Latn.yaml' 2026-05-11T15:53:27,541 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ewe_Latn.yaml' 2026-05-11T15:53:27,542 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fon_Latn.yaml' 2026-05-11T15:53:27,543 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fra_Latn.yaml' 2026-05-11T15:53:27,544 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-fuv_Latn.yaml' 2026-05-11T15:53:27,546 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-gaz_Latn.yaml' 2026-05-11T15:53:27,547 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-hau_Latn.yaml' 2026-05-11T15:53:27,549 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ibo_Latn.yaml' 2026-05-11T15:53:27,550 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kab_Latn.yaml' 2026-05-11T15:53:27,551 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kam_Latn.yaml' 2026-05-11T15:53:27,552 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kbp_Latn.yaml' 2026-05-11T15:53:27,553 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kea_Latn.yaml' 2026-05-11T15:53:27,555 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kik_Latn.yaml' 2026-05-11T15:53:27,556 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kin_Latn.yaml' 2026-05-11T15:53:27,557 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kmb_Latn.yaml' 2026-05-11T15:53:27,558 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Arab.yaml' 2026-05-11T15:53:27,560 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-knc_Latn.yaml' 2026-05-11T15:53:27,561 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-kon_Latn.yaml' 2026-05-11T15:53:27,562 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lin_Latn.yaml' 2026-05-11T15:53:27,563 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lua_Latn.yaml' 2026-05-11T15:53:27,565 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-lug_Latn.yaml' 2026-05-11T15:53:27,566 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-luo_Latn.yaml' 2026-05-11T15:53:27,567 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-mos_Latn.yaml' 2026-05-11T15:53:27,568 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nso_Latn.yaml' 2026-05-11T15:53:27,570 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nus_Latn.yaml' 2026-05-11T15:53:27,571 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-nya_Latn.yaml' 2026-05-11T15:53:27,572 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-plt_Latn.yaml' 2026-05-11T15:53:27,573 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-run_Latn.yaml' 2026-05-11T15:53:27,574 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sag_Latn.yaml' 2026-05-11T15:53:27,576 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sna_Latn.yaml' 2026-05-11T15:53:27,577 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-som_Latn.yaml' 2026-05-11T15:53:27,578 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sot_Latn.yaml' 2026-05-11T15:53:27,580 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-ssw_Latn.yaml' 2026-05-11T15:53:27,581 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-sun_Latn.yaml' 2026-05-11T15:53:27,583 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-swh_Latn.yaml' 2026-05-11T15:53:27,585 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Latn.yaml' 2026-05-11T15:53:27,586 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-taq_Tfng.yaml' 2026-05-11T15:53:27,588 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tir_Ethi.yaml' 2026-05-11T15:53:27,590 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tsn_Latn.yaml' 2026-05-11T15:53:27,591 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tso_Latn.yaml' 2026-05-11T15:53:27,593 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tum_Latn.yaml' 2026-05-11T15:53:27,595 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-twi_Latn.yaml' 2026-05-11T15:53:27,596 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-tzm_Tfng.yaml' 2026-05-11T15:53:27,598 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-umb_Latn.yaml' 2026-05-11T15:53:27,599 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-wol_Latn.yaml' 2026-05-11T15:53:27,601 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-xho_Latn.yaml' 2026-05-11T15:53:27,602 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-yor_Latn.yaml' 2026-05-11T15:53:27,604 adding 'lm_eval/tasks/afrobench/flores/prompt_3/english-african/flores_eng_Latn-zul_Latn.yaml' 2026-05-11T15:53:27,606 adding 'lm_eval/tasks/afrobench/injongointent/README.md' 2026-05-11T15:53:27,608 adding 'lm_eval/tasks/afrobench/injongointent/gen_utils.py' 2026-05-11T15:53:27,610 adding 'lm_eval/tasks/afrobench/injongointent/injongointent.yaml' 2026-05-11T15:53:27,612 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent' 2026-05-11T15:53:27,614 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_amh.yaml' 2026-05-11T15:53:27,615 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_eng.yaml' 2026-05-11T15:53:27,617 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ewe.yaml' 2026-05-11T15:53:27,618 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_hau.yaml' 2026-05-11T15:53:27,620 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_ibo.yaml' 2026-05-11T15:53:27,622 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_kin.yaml' 2026-05-11T15:53:27,623 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lin.yaml' 2026-05-11T15:53:27,625 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_lug.yaml' 2026-05-11T15:53:27,626 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_orm.yaml' 2026-05-11T15:53:27,628 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sna.yaml' 2026-05-11T15:53:27,630 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_sot.yaml' 2026-05-11T15:53:27,631 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_swa.yaml' 2026-05-11T15:53:27,633 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_twi.yaml' 2026-05-11T15:53:27,634 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_wol.yaml' 2026-05-11T15:53:27,636 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_xho.yaml' 2026-05-11T15:53:27,638 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_yor.yaml' 2026-05-11T15:53:27,639 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/injongointent_zul.yaml' 2026-05-11T15:53:27,641 adding 'lm_eval/tasks/afrobench/injongointent/prompt_1/utils.py' 2026-05-11T15:53:27,643 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent' 2026-05-11T15:53:27,645 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_amh.yaml' 2026-05-11T15:53:27,647 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_eng.yaml' 2026-05-11T15:53:27,648 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ewe.yaml' 2026-05-11T15:53:27,650 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_hau.yaml' 2026-05-11T15:53:27,652 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_ibo.yaml' 2026-05-11T15:53:27,653 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_kin.yaml' 2026-05-11T15:53:27,655 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lin.yaml' 2026-05-11T15:53:27,656 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_lug.yaml' 2026-05-11T15:53:27,658 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_orm.yaml' 2026-05-11T15:53:27,660 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sna.yaml' 2026-05-11T15:53:27,662 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_sot.yaml' 2026-05-11T15:53:27,663 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_swa.yaml' 2026-05-11T15:53:27,665 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_twi.yaml' 2026-05-11T15:53:27,667 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_wol.yaml' 2026-05-11T15:53:27,669 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_xho.yaml' 2026-05-11T15:53:27,670 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_yor.yaml' 2026-05-11T15:53:27,672 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/injongointent_zul.yaml' 2026-05-11T15:53:27,674 adding 'lm_eval/tasks/afrobench/injongointent/prompt_2/utils.py' 2026-05-11T15:53:27,677 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent' 2026-05-11T15:53:27,679 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_amh.yaml' 2026-05-11T15:53:27,680 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_eng.yaml' 2026-05-11T15:53:27,682 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ewe.yaml' 2026-05-11T15:53:27,684 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_hau.yaml' 2026-05-11T15:53:27,686 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_ibo.yaml' 2026-05-11T15:53:27,688 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_kin.yaml' 2026-05-11T15:53:27,689 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lin.yaml' 2026-05-11T15:53:27,691 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_lug.yaml' 2026-05-11T15:53:27,693 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_orm.yaml' 2026-05-11T15:53:27,694 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sna.yaml' 2026-05-11T15:53:27,696 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_sot.yaml' 2026-05-11T15:53:27,697 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_swa.yaml' 2026-05-11T15:53:27,699 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_twi.yaml' 2026-05-11T15:53:27,700 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_wol.yaml' 2026-05-11T15:53:27,702 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_xho.yaml' 2026-05-11T15:53:27,703 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_yor.yaml' 2026-05-11T15:53:27,705 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/injongointent_zul.yaml' 2026-05-11T15:53:27,706 adding 'lm_eval/tasks/afrobench/injongointent/prompt_3/utils.py' 2026-05-11T15:53:27,709 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent' 2026-05-11T15:53:27,710 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_amh.yaml' 2026-05-11T15:53:27,712 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_eng.yaml' 2026-05-11T15:53:27,713 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ewe.yaml' 2026-05-11T15:53:27,715 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_hau.yaml' 2026-05-11T15:53:27,716 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_ibo.yaml' 2026-05-11T15:53:27,718 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_kin.yaml' 2026-05-11T15:53:27,719 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lin.yaml' 2026-05-11T15:53:27,721 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_lug.yaml' 2026-05-11T15:53:27,723 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_orm.yaml' 2026-05-11T15:53:27,725 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sna.yaml' 2026-05-11T15:53:27,726 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_sot.yaml' 2026-05-11T15:53:27,728 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_swa.yaml' 2026-05-11T15:53:27,729 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_twi.yaml' 2026-05-11T15:53:27,731 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_wol.yaml' 2026-05-11T15:53:27,733 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_xho.yaml' 2026-05-11T15:53:27,734 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_yor.yaml' 2026-05-11T15:53:27,736 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/injongointent_zul.yaml' 2026-05-11T15:53:27,737 adding 'lm_eval/tasks/afrobench/injongointent/prompt_4/utils.py' 2026-05-11T15:53:27,740 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent' 2026-05-11T15:53:27,742 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_amh.yaml' 2026-05-11T15:53:27,743 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_eng.yaml' 2026-05-11T15:53:27,745 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ewe.yaml' 2026-05-11T15:53:27,747 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_hau.yaml' 2026-05-11T15:53:27,749 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_ibo.yaml' 2026-05-11T15:53:27,750 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_kin.yaml' 2026-05-11T15:53:27,752 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lin.yaml' 2026-05-11T15:53:27,754 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_lug.yaml' 2026-05-11T15:53:27,755 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_orm.yaml' 2026-05-11T15:53:27,757 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sna.yaml' 2026-05-11T15:53:27,759 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_sot.yaml' 2026-05-11T15:53:27,761 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_swa.yaml' 2026-05-11T15:53:27,762 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_twi.yaml' 2026-05-11T15:53:27,764 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_wol.yaml' 2026-05-11T15:53:27,766 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_xho.yaml' 2026-05-11T15:53:27,767 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_yor.yaml' 2026-05-11T15:53:27,769 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/injongointent_zul.yaml' 2026-05-11T15:53:27,771 adding 'lm_eval/tasks/afrobench/injongointent/prompt_5/utils.py' 2026-05-11T15:53:27,774 adding 'lm_eval/tasks/afrobench/mafand/README.md' 2026-05-11T15:53:27,776 adding 'lm_eval/tasks/afrobench/mafand/gen_utils.py' 2026-05-11T15:53:27,777 adding 'lm_eval/tasks/afrobench/mafand/mafand.yaml' 2026-05-11T15:53:27,780 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand' 2026-05-11T15:53:27,781 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_amh-en.yaml' 2026-05-11T15:53:27,782 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bam-fr.yaml' 2026-05-11T15:53:27,784 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_bbj-fr.yaml' 2026-05-11T15:53:27,785 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ewe-fr.yaml' 2026-05-11T15:53:27,786 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_fon-fr.yaml' 2026-05-11T15:53:27,787 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_hau-en.yaml' 2026-05-11T15:53:27,788 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_ibo-en.yaml' 2026-05-11T15:53:27,790 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_kin-en.yaml' 2026-05-11T15:53:27,791 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_lug-en.yaml' 2026-05-11T15:53:27,792 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_luo-en.yaml' 2026-05-11T15:53:27,793 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_mos-fr.yaml' 2026-05-11T15:53:27,795 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_nya-en.yaml' 2026-05-11T15:53:27,796 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_pcm-en.yaml' 2026-05-11T15:53:27,797 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_sna-en.yaml' 2026-05-11T15:53:27,798 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_swa-en.yaml' 2026-05-11T15:53:27,799 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_tsn-en.yaml' 2026-05-11T15:53:27,800 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_twi-en.yaml' 2026-05-11T15:53:27,802 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_wol-fr.yaml' 2026-05-11T15:53:27,803 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_xho-en.yaml' 2026-05-11T15:53:27,804 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_yor-en.yaml' 2026-05-11T15:53:27,805 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/mafand_zul-en.yaml' 2026-05-11T15:53:27,807 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/african-english/utils.py' 2026-05-11T15:53:27,809 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand' 2026-05-11T15:53:27,810 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-amh.yaml' 2026-05-11T15:53:27,811 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-hau.yaml' 2026-05-11T15:53:27,813 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-ibo.yaml' 2026-05-11T15:53:27,814 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-kin.yaml' 2026-05-11T15:53:27,815 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-lug.yaml' 2026-05-11T15:53:27,816 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-luo.yaml' 2026-05-11T15:53:27,817 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-nya.yaml' 2026-05-11T15:53:27,818 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-pcm.yaml' 2026-05-11T15:53:27,819 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-sna.yaml' 2026-05-11T15:53:27,820 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-swa.yaml' 2026-05-11T15:53:27,822 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-tsn.yaml' 2026-05-11T15:53:27,823 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-twi.yaml' 2026-05-11T15:53:27,824 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-xho.yaml' 2026-05-11T15:53:27,825 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-yor.yaml' 2026-05-11T15:53:27,826 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_en-zul.yaml' 2026-05-11T15:53:27,827 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bam.yaml' 2026-05-11T15:53:27,828 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-bbj.yaml' 2026-05-11T15:53:27,830 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-ewe.yaml' 2026-05-11T15:53:27,831 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-fon.yaml' 2026-05-11T15:53:27,832 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-mos.yaml' 2026-05-11T15:53:27,833 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/mafand_fr-wol.yaml' 2026-05-11T15:53:27,834 adding 'lm_eval/tasks/afrobench/mafand/prompt_1/english-african/utils.py' 2026-05-11T15:53:27,837 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand' 2026-05-11T15:53:27,838 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_amh-en.yaml' 2026-05-11T15:53:27,839 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bam-fr.yaml' 2026-05-11T15:53:27,840 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_bbj-fr.yaml' 2026-05-11T15:53:27,841 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ewe-fr.yaml' 2026-05-11T15:53:27,842 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_fon-fr.yaml' 2026-05-11T15:53:27,843 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_hau-en.yaml' 2026-05-11T15:53:27,845 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_ibo-en.yaml' 2026-05-11T15:53:27,846 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_kin-en.yaml' 2026-05-11T15:53:27,847 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_lug-en.yaml' 2026-05-11T15:53:27,848 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_luo-en.yaml' 2026-05-11T15:53:27,849 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_mos-fr.yaml' 2026-05-11T15:53:27,851 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_nya-en.yaml' 2026-05-11T15:53:27,852 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_pcm-en.yaml' 2026-05-11T15:53:27,853 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_sna-en.yaml' 2026-05-11T15:53:27,854 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_swa-en.yaml' 2026-05-11T15:53:27,855 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_tsn-en.yaml' 2026-05-11T15:53:27,856 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_twi-en.yaml' 2026-05-11T15:53:27,857 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_wol-fr.yaml' 2026-05-11T15:53:27,859 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_xho-en.yaml' 2026-05-11T15:53:27,860 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_yor-en.yaml' 2026-05-11T15:53:27,861 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/mafand_zul-en.yaml' 2026-05-11T15:53:27,862 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/african-english/utils.py' 2026-05-11T15:53:27,864 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand' 2026-05-11T15:53:27,865 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-amh.yaml' 2026-05-11T15:53:27,866 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-hau.yaml' 2026-05-11T15:53:27,867 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-ibo.yaml' 2026-05-11T15:53:27,868 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-kin.yaml' 2026-05-11T15:53:27,869 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-lug.yaml' 2026-05-11T15:53:27,871 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-luo.yaml' 2026-05-11T15:53:27,872 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-nya.yaml' 2026-05-11T15:53:27,873 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-pcm.yaml' 2026-05-11T15:53:27,874 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-sna.yaml' 2026-05-11T15:53:27,875 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-swa.yaml' 2026-05-11T15:53:27,876 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-tsn.yaml' 2026-05-11T15:53:27,877 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-twi.yaml' 2026-05-11T15:53:27,878 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-xho.yaml' 2026-05-11T15:53:27,880 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-yor.yaml' 2026-05-11T15:53:27,881 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_en-zul.yaml' 2026-05-11T15:53:27,882 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bam.yaml' 2026-05-11T15:53:27,883 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-bbj.yaml' 2026-05-11T15:53:27,884 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-ewe.yaml' 2026-05-11T15:53:27,885 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-fon.yaml' 2026-05-11T15:53:27,886 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-mos.yaml' 2026-05-11T15:53:27,887 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/mafand_fr-wol.yaml' 2026-05-11T15:53:27,889 adding 'lm_eval/tasks/afrobench/mafand/prompt_2/english-african/utils.py' 2026-05-11T15:53:27,891 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand' 2026-05-11T15:53:27,892 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_amh-en.yaml' 2026-05-11T15:53:27,894 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bam-fr.yaml' 2026-05-11T15:53:27,895 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_bbj-fr.yaml' 2026-05-11T15:53:27,896 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ewe-fr.yaml' 2026-05-11T15:53:27,897 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_fon-fr.yaml' 2026-05-11T15:53:27,898 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_hau-en.yaml' 2026-05-11T15:53:27,899 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_ibo-en.yaml' 2026-05-11T15:53:27,901 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_kin-en.yaml' 2026-05-11T15:53:27,902 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_lug-en.yaml' 2026-05-11T15:53:27,903 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_luo-en.yaml' 2026-05-11T15:53:27,904 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_mos-fr.yaml' 2026-05-11T15:53:27,905 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_nya-en.yaml' 2026-05-11T15:53:27,906 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_pcm-en.yaml' 2026-05-11T15:53:27,907 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_sna-en.yaml' 2026-05-11T15:53:27,908 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_swa-en.yaml' 2026-05-11T15:53:27,910 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_tsn-en.yaml' 2026-05-11T15:53:27,911 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_twi-en.yaml' 2026-05-11T15:53:27,912 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_wol-fr.yaml' 2026-05-11T15:53:27,913 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_xho-en.yaml' 2026-05-11T15:53:27,914 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_yor-en.yaml' 2026-05-11T15:53:27,915 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/mafand_zul-en.yaml' 2026-05-11T15:53:27,917 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/african-english/utils.py' 2026-05-11T15:53:27,919 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand' 2026-05-11T15:53:27,920 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-amh.yaml' 2026-05-11T15:53:27,921 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-hau.yaml' 2026-05-11T15:53:27,922 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-ibo.yaml' 2026-05-11T15:53:27,923 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-kin.yaml' 2026-05-11T15:53:27,924 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-lug.yaml' 2026-05-11T15:53:27,925 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-luo.yaml' 2026-05-11T15:53:27,926 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-nya.yaml' 2026-05-11T15:53:27,928 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-pcm.yaml' 2026-05-11T15:53:27,929 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-sna.yaml' 2026-05-11T15:53:27,930 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-swa.yaml' 2026-05-11T15:53:27,931 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-tsn.yaml' 2026-05-11T15:53:27,932 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-twi.yaml' 2026-05-11T15:53:27,933 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-xho.yaml' 2026-05-11T15:53:27,934 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-yor.yaml' 2026-05-11T15:53:27,935 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_en-zul.yaml' 2026-05-11T15:53:27,937 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bam.yaml' 2026-05-11T15:53:27,938 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-bbj.yaml' 2026-05-11T15:53:27,939 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-ewe.yaml' 2026-05-11T15:53:27,940 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-fon.yaml' 2026-05-11T15:53:27,941 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-mos.yaml' 2026-05-11T15:53:27,942 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/mafand_fr-wol.yaml' 2026-05-11T15:53:27,944 adding 'lm_eval/tasks/afrobench/mafand/prompt_3/english-african/utils.py' 2026-05-11T15:53:27,946 adding 'lm_eval/tasks/afrobench/masakhaner/README.md' 2026-05-11T15:53:27,947 adding 'lm_eval/tasks/afrobench/masakhaner/gen_utils.py' 2026-05-11T15:53:27,948 adding 'lm_eval/tasks/afrobench/masakhaner/masakhaner.yaml' 2026-05-11T15:53:27,950 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner' 2026-05-11T15:53:27,951 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_am.yaml' 2026-05-11T15:53:27,952 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bbj.yaml' 2026-05-11T15:53:27,953 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_bm.yaml' 2026-05-11T15:53:27,955 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ee.yaml' 2026-05-11T15:53:27,956 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ha.yaml' 2026-05-11T15:53:27,957 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ig.yaml' 2026-05-11T15:53:27,958 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_lg.yaml' 2026-05-11T15:53:27,959 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_luo.yaml' 2026-05-11T15:53:27,960 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_mos.yaml' 2026-05-11T15:53:27,961 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_ny.yaml' 2026-05-11T15:53:27,962 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_pcm.yaml' 2026-05-11T15:53:27,964 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_rw.yaml' 2026-05-11T15:53:27,965 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sn.yaml' 2026-05-11T15:53:27,966 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_sw.yaml' 2026-05-11T15:53:27,967 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tn.yaml' 2026-05-11T15:53:27,969 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_tw.yaml' 2026-05-11T15:53:27,970 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_wo.yaml' 2026-05-11T15:53:27,971 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_xh.yaml' 2026-05-11T15:53:27,972 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_yo.yaml' 2026-05-11T15:53:27,973 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/masakhaner_zu.yaml' 2026-05-11T15:53:27,975 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_1/utils.py' 2026-05-11T15:53:27,977 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner' 2026-05-11T15:53:27,978 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_am.yaml' 2026-05-11T15:53:27,979 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bbj.yaml' 2026-05-11T15:53:27,980 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_bm.yaml' 2026-05-11T15:53:27,982 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ee.yaml' 2026-05-11T15:53:27,983 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ha.yaml' 2026-05-11T15:53:27,984 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ig.yaml' 2026-05-11T15:53:27,985 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_lg.yaml' 2026-05-11T15:53:27,987 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_luo.yaml' 2026-05-11T15:53:27,988 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_mos.yaml' 2026-05-11T15:53:27,989 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_ny.yaml' 2026-05-11T15:53:27,990 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_pcm.yaml' 2026-05-11T15:53:27,992 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_rw.yaml' 2026-05-11T15:53:27,993 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sn.yaml' 2026-05-11T15:53:27,994 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_sw.yaml' 2026-05-11T15:53:27,996 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tn.yaml' 2026-05-11T15:53:27,997 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_tw.yaml' 2026-05-11T15:53:27,998 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_wo.yaml' 2026-05-11T15:53:28,000 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_xh.yaml' 2026-05-11T15:53:28,001 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_yo.yaml' 2026-05-11T15:53:28,003 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/masakhaner_zu.yaml' 2026-05-11T15:53:28,004 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_2/utils.py' 2026-05-11T15:53:28,006 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner' 2026-05-11T15:53:28,007 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_am.yaml' 2026-05-11T15:53:28,008 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bbj.yaml' 2026-05-11T15:53:28,010 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_bm.yaml' 2026-05-11T15:53:28,011 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ee.yaml' 2026-05-11T15:53:28,012 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ha.yaml' 2026-05-11T15:53:28,013 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ig.yaml' 2026-05-11T15:53:28,014 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_lg.yaml' 2026-05-11T15:53:28,016 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_luo.yaml' 2026-05-11T15:53:28,017 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_mos.yaml' 2026-05-11T15:53:28,018 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_ny.yaml' 2026-05-11T15:53:28,019 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_pcm.yaml' 2026-05-11T15:53:28,021 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_rw.yaml' 2026-05-11T15:53:28,022 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sn.yaml' 2026-05-11T15:53:28,023 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_sw.yaml' 2026-05-11T15:53:28,024 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tn.yaml' 2026-05-11T15:53:28,025 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_tw.yaml' 2026-05-11T15:53:28,027 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_wo.yaml' 2026-05-11T15:53:28,028 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_xh.yaml' 2026-05-11T15:53:28,029 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_yo.yaml' 2026-05-11T15:53:28,030 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/masakhaner_zu.yaml' 2026-05-11T15:53:28,032 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_3/utils.py' 2026-05-11T15:53:28,034 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner' 2026-05-11T15:53:28,035 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_am.yaml' 2026-05-11T15:53:28,036 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bbj.yaml' 2026-05-11T15:53:28,038 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_bm.yaml' 2026-05-11T15:53:28,039 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ee.yaml' 2026-05-11T15:53:28,040 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ha.yaml' 2026-05-11T15:53:28,041 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ig.yaml' 2026-05-11T15:53:28,043 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_lg.yaml' 2026-05-11T15:53:28,044 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_luo.yaml' 2026-05-11T15:53:28,045 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_mos.yaml' 2026-05-11T15:53:28,046 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_ny.yaml' 2026-05-11T15:53:28,048 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_pcm.yaml' 2026-05-11T15:53:28,049 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_rw.yaml' 2026-05-11T15:53:28,050 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sn.yaml' 2026-05-11T15:53:28,052 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_sw.yaml' 2026-05-11T15:53:28,053 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tn.yaml' 2026-05-11T15:53:28,054 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_tw.yaml' 2026-05-11T15:53:28,056 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_wo.yaml' 2026-05-11T15:53:28,057 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_xh.yaml' 2026-05-11T15:53:28,058 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_yo.yaml' 2026-05-11T15:53:28,060 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/masakhaner_zu.yaml' 2026-05-11T15:53:28,061 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_4/utils.py' 2026-05-11T15:53:28,063 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner' 2026-05-11T15:53:28,064 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_am.yaml' 2026-05-11T15:53:28,065 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bbj.yaml' 2026-05-11T15:53:28,066 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_bm.yaml' 2026-05-11T15:53:28,068 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ee.yaml' 2026-05-11T15:53:28,069 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ha.yaml' 2026-05-11T15:53:28,070 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ig.yaml' 2026-05-11T15:53:28,071 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_lg.yaml' 2026-05-11T15:53:28,073 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_luo.yaml' 2026-05-11T15:53:28,074 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_mos.yaml' 2026-05-11T15:53:28,075 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_ny.yaml' 2026-05-11T15:53:28,076 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_pcm.yaml' 2026-05-11T15:53:28,078 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_rw.yaml' 2026-05-11T15:53:28,079 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sn.yaml' 2026-05-11T15:53:28,080 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_sw.yaml' 2026-05-11T15:53:28,081 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tn.yaml' 2026-05-11T15:53:28,083 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_tw.yaml' 2026-05-11T15:53:28,084 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_wo.yaml' 2026-05-11T15:53:28,085 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_xh.yaml' 2026-05-11T15:53:28,086 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_yo.yaml' 2026-05-11T15:53:28,088 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/masakhaner_zu.yaml' 2026-05-11T15:53:28,089 adding 'lm_eval/tasks/afrobench/masakhaner/prompt_5/utils.py' 2026-05-11T15:53:28,091 adding 'lm_eval/tasks/afrobench/masakhanews/README.md' 2026-05-11T15:53:28,093 adding 'lm_eval/tasks/afrobench/masakhanews/masakhanews.yaml' 2026-05-11T15:53:28,094 adding 'lm_eval/tasks/afrobench/masakhanews/utils.py' 2026-05-11T15:53:28,096 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews' 2026-05-11T15:53:28,097 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_amh.yaml' 2026-05-11T15:53:28,099 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_eng.yaml' 2026-05-11T15:53:28,100 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_fra.yaml' 2026-05-11T15:53:28,101 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_hau.yaml' 2026-05-11T15:53:28,102 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_ibo.yaml' 2026-05-11T15:53:28,104 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lin.yaml' 2026-05-11T15:53:28,105 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_lug.yaml' 2026-05-11T15:53:28,106 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_orm.yaml' 2026-05-11T15:53:28,108 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_pcm.yaml' 2026-05-11T15:53:28,109 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_run.yaml' 2026-05-11T15:53:28,110 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_sna.yaml' 2026-05-11T15:53:28,111 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_som.yaml' 2026-05-11T15:53:28,113 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_swa.yaml' 2026-05-11T15:53:28,114 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_tir.yaml' 2026-05-11T15:53:28,115 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_xho.yaml' 2026-05-11T15:53:28,116 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/masakhanews_yor.yaml' 2026-05-11T15:53:28,117 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_1/utils.py' 2026-05-11T15:53:28,119 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews' 2026-05-11T15:53:28,121 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_amh.yaml' 2026-05-11T15:53:28,122 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_eng.yaml' 2026-05-11T15:53:28,123 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_fra.yaml' 2026-05-11T15:53:28,124 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_hau.yaml' 2026-05-11T15:53:28,125 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_ibo.yaml' 2026-05-11T15:53:28,126 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lin.yaml' 2026-05-11T15:53:28,128 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_lug.yaml' 2026-05-11T15:53:28,129 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_orm.yaml' 2026-05-11T15:53:28,130 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_pcm.yaml' 2026-05-11T15:53:28,131 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_run.yaml' 2026-05-11T15:53:28,133 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_sna.yaml' 2026-05-11T15:53:28,134 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_som.yaml' 2026-05-11T15:53:28,135 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_swa.yaml' 2026-05-11T15:53:28,136 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_tir.yaml' 2026-05-11T15:53:28,137 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_xho.yaml' 2026-05-11T15:53:28,138 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/masakhanews_yor.yaml' 2026-05-11T15:53:28,140 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_2/utils.py' 2026-05-11T15:53:28,142 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews' 2026-05-11T15:53:28,143 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_amh.yaml' 2026-05-11T15:53:28,144 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_eng.yaml' 2026-05-11T15:53:28,145 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_fra.yaml' 2026-05-11T15:53:28,147 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_hau.yaml' 2026-05-11T15:53:28,148 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_ibo.yaml' 2026-05-11T15:53:28,149 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lin.yaml' 2026-05-11T15:53:28,150 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_lug.yaml' 2026-05-11T15:53:28,152 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_orm.yaml' 2026-05-11T15:53:28,153 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_pcm.yaml' 2026-05-11T15:53:28,154 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_run.yaml' 2026-05-11T15:53:28,156 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_sna.yaml' 2026-05-11T15:53:28,157 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_som.yaml' 2026-05-11T15:53:28,159 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_swa.yaml' 2026-05-11T15:53:28,160 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_tir.yaml' 2026-05-11T15:53:28,161 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_xho.yaml' 2026-05-11T15:53:28,163 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/masakhanews_yor.yaml' 2026-05-11T15:53:28,164 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_3/utils.py' 2026-05-11T15:53:28,166 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews' 2026-05-11T15:53:28,167 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_amh.yaml' 2026-05-11T15:53:28,168 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_eng.yaml' 2026-05-11T15:53:28,170 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_fra.yaml' 2026-05-11T15:53:28,171 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_hau.yaml' 2026-05-11T15:53:28,172 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_ibo.yaml' 2026-05-11T15:53:28,173 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lin.yaml' 2026-05-11T15:53:28,174 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_lug.yaml' 2026-05-11T15:53:28,175 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_orm.yaml' 2026-05-11T15:53:28,177 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_pcm.yaml' 2026-05-11T15:53:28,178 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_run.yaml' 2026-05-11T15:53:28,179 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_sna.yaml' 2026-05-11T15:53:28,180 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_som.yaml' 2026-05-11T15:53:28,181 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_swa.yaml' 2026-05-11T15:53:28,183 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_tir.yaml' 2026-05-11T15:53:28,184 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_xho.yaml' 2026-05-11T15:53:28,185 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/masakhanews_yor.yaml' 2026-05-11T15:53:28,186 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_4/utils.py' 2026-05-11T15:53:28,188 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews' 2026-05-11T15:53:28,189 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_amh.yaml' 2026-05-11T15:53:28,191 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_eng.yaml' 2026-05-11T15:53:28,192 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_fra.yaml' 2026-05-11T15:53:28,193 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_hau.yaml' 2026-05-11T15:53:28,195 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_ibo.yaml' 2026-05-11T15:53:28,196 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lin.yaml' 2026-05-11T15:53:28,197 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_lug.yaml' 2026-05-11T15:53:28,198 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_orm.yaml' 2026-05-11T15:53:28,200 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_pcm.yaml' 2026-05-11T15:53:28,201 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_run.yaml' 2026-05-11T15:53:28,202 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_sna.yaml' 2026-05-11T15:53:28,204 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_som.yaml' 2026-05-11T15:53:28,205 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_swa.yaml' 2026-05-11T15:53:28,207 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_tir.yaml' 2026-05-11T15:53:28,208 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_xho.yaml' 2026-05-11T15:53:28,210 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/masakhanews_yor.yaml' 2026-05-11T15:53:28,211 adding 'lm_eval/tasks/afrobench/masakhanews/prompt_5/utils.py' 2026-05-11T15:53:28,213 adding 'lm_eval/tasks/afrobench/masakhapos/README.md' 2026-05-11T15:53:28,215 adding 'lm_eval/tasks/afrobench/masakhapos/gen_utils.py' 2026-05-11T15:53:28,216 adding 'lm_eval/tasks/afrobench/masakhapos/masakhapos.yaml' 2026-05-11T15:53:28,218 adding 'lm_eval/tasks/afrobench/masakhapos/utils.py' 2026-05-11T15:53:28,220 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bam.yaml' 2026-05-11T15:53:28,221 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_bbj.yaml' 2026-05-11T15:53:28,223 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ewe.yaml' 2026-05-11T15:53:28,224 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_fon.yaml' 2026-05-11T15:53:28,225 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_hau.yaml' 2026-05-11T15:53:28,226 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_ibo.yaml' 2026-05-11T15:53:28,228 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_kin.yaml' 2026-05-11T15:53:28,229 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_lug.yaml' 2026-05-11T15:53:28,230 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_luo.yaml' 2026-05-11T15:53:28,232 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_mos.yaml' 2026-05-11T15:53:28,233 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_nya.yaml' 2026-05-11T15:53:28,234 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_pcm.yaml' 2026-05-11T15:53:28,235 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_sna.yaml' 2026-05-11T15:53:28,237 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_swa.yaml' 2026-05-11T15:53:28,238 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_tsn.yaml' 2026-05-11T15:53:28,240 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_twi.yaml' 2026-05-11T15:53:28,241 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_wol.yaml' 2026-05-11T15:53:28,242 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_xho.yaml' 2026-05-11T15:53:28,243 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yaml' 2026-05-11T15:53:28,244 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_yor.yaml' 2026-05-11T15:53:28,245 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/masakhapos_zul.yaml' 2026-05-11T15:53:28,247 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_1/utils.py' 2026-05-11T15:53:28,249 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bam.yaml' 2026-05-11T15:53:28,250 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_bbj.yaml' 2026-05-11T15:53:28,252 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ewe.yaml' 2026-05-11T15:53:28,253 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_fon.yaml' 2026-05-11T15:53:28,254 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_hau.yaml' 2026-05-11T15:53:28,255 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_ibo.yaml' 2026-05-11T15:53:28,257 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_kin.yaml' 2026-05-11T15:53:28,258 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_lug.yaml' 2026-05-11T15:53:28,259 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_luo.yaml' 2026-05-11T15:53:28,260 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_mos.yaml' 2026-05-11T15:53:28,262 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_nya.yaml' 2026-05-11T15:53:28,263 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_pcm.yaml' 2026-05-11T15:53:28,264 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_sna.yaml' 2026-05-11T15:53:28,265 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_swa.yaml' 2026-05-11T15:53:28,267 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_tsn.yaml' 2026-05-11T15:53:28,268 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_twi.yaml' 2026-05-11T15:53:28,269 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_wol.yaml' 2026-05-11T15:53:28,270 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_xho.yaml' 2026-05-11T15:53:28,272 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yaml' 2026-05-11T15:53:28,273 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_yor.yaml' 2026-05-11T15:53:28,274 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/masakhapos_zul.yaml' 2026-05-11T15:53:28,276 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_2/utils.py' 2026-05-11T15:53:28,278 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bam.yaml' 2026-05-11T15:53:28,279 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_bbj.yaml' 2026-05-11T15:53:28,280 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ewe.yaml' 2026-05-11T15:53:28,281 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_fon.yaml' 2026-05-11T15:53:28,282 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_hau.yaml' 2026-05-11T15:53:28,284 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_ibo.yaml' 2026-05-11T15:53:28,285 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_kin.yaml' 2026-05-11T15:53:28,286 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_lug.yaml' 2026-05-11T15:53:28,287 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_luo.yaml' 2026-05-11T15:53:28,288 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_mos.yaml' 2026-05-11T15:53:28,290 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_nya.yaml' 2026-05-11T15:53:28,291 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_pcm.yaml' 2026-05-11T15:53:28,292 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_sna.yaml' 2026-05-11T15:53:28,293 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_swa.yaml' 2026-05-11T15:53:28,295 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_tsn.yaml' 2026-05-11T15:53:28,296 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_twi.yaml' 2026-05-11T15:53:28,297 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_wol.yaml' 2026-05-11T15:53:28,298 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_xho.yaml' 2026-05-11T15:53:28,300 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yaml' 2026-05-11T15:53:28,301 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_yor.yaml' 2026-05-11T15:53:28,302 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/masakhapos_zul.yaml' 2026-05-11T15:53:28,304 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_3/utils.py' 2026-05-11T15:53:28,306 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bam.yaml' 2026-05-11T15:53:28,307 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_bbj.yaml' 2026-05-11T15:53:28,308 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ewe.yaml' 2026-05-11T15:53:28,310 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_fon.yaml' 2026-05-11T15:53:28,311 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_hau.yaml' 2026-05-11T15:53:28,312 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_ibo.yaml' 2026-05-11T15:53:28,313 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_kin.yaml' 2026-05-11T15:53:28,315 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_lug.yaml' 2026-05-11T15:53:28,316 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_luo.yaml' 2026-05-11T15:53:28,317 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_mos.yaml' 2026-05-11T15:53:28,318 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_nya.yaml' 2026-05-11T15:53:28,320 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_pcm.yaml' 2026-05-11T15:53:28,321 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_sna.yaml' 2026-05-11T15:53:28,322 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_swa.yaml' 2026-05-11T15:53:28,324 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_tsn.yaml' 2026-05-11T15:53:28,325 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_twi.yaml' 2026-05-11T15:53:28,326 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_wol.yaml' 2026-05-11T15:53:28,328 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_xho.yaml' 2026-05-11T15:53:28,329 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yaml' 2026-05-11T15:53:28,330 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_yor.yaml' 2026-05-11T15:53:28,331 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/masakhapos_zul.yaml' 2026-05-11T15:53:28,332 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_4/utils.py' 2026-05-11T15:53:28,334 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bam.yaml' 2026-05-11T15:53:28,335 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_bbj.yaml' 2026-05-11T15:53:28,337 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ewe.yaml' 2026-05-11T15:53:28,338 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_fon.yaml' 2026-05-11T15:53:28,339 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_hau.yaml' 2026-05-11T15:53:28,340 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_ibo.yaml' 2026-05-11T15:53:28,341 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_kin.yaml' 2026-05-11T15:53:28,343 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_lug.yaml' 2026-05-11T15:53:28,344 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_luo.yaml' 2026-05-11T15:53:28,345 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_mos.yaml' 2026-05-11T15:53:28,346 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_nya.yaml' 2026-05-11T15:53:28,348 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_pcm.yaml' 2026-05-11T15:53:28,349 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_sna.yaml' 2026-05-11T15:53:28,351 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_swa.yaml' 2026-05-11T15:53:28,352 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_tsn.yaml' 2026-05-11T15:53:28,353 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_twi.yaml' 2026-05-11T15:53:28,354 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_wol.yaml' 2026-05-11T15:53:28,356 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_xho.yaml' 2026-05-11T15:53:28,357 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yaml' 2026-05-11T15:53:28,358 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_yor.yaml' 2026-05-11T15:53:28,359 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/masakhapos_zul.yaml' 2026-05-11T15:53:28,361 adding 'lm_eval/tasks/afrobench/masakhapos/prompt_5/utils.py' 2026-05-11T15:53:28,362 adding 'lm_eval/tasks/afrobench/naijarc/README.md' 2026-05-11T15:53:28,364 adding 'lm_eval/tasks/afrobench/naijarc/naijarc.yaml' 2026-05-11T15:53:28,365 adding 'lm_eval/tasks/afrobench/naijarc/utils.py' 2026-05-11T15:53:28,367 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc' 2026-05-11T15:53:28,368 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_hau.yaml' 2026-05-11T15:53:28,369 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_ibo.yaml' 2026-05-11T15:53:28,370 adding 'lm_eval/tasks/afrobench/naijarc/prompt_1/naijarc_yor.yaml' 2026-05-11T15:53:28,372 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc' 2026-05-11T15:53:28,373 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_hau.yaml' 2026-05-11T15:53:28,375 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_ibo.yaml' 2026-05-11T15:53:28,376 adding 'lm_eval/tasks/afrobench/naijarc/prompt_2/naijarc_yor.yaml' 2026-05-11T15:53:28,378 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc' 2026-05-11T15:53:28,379 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_hau.yaml' 2026-05-11T15:53:28,380 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_ibo.yaml' 2026-05-11T15:53:28,381 adding 'lm_eval/tasks/afrobench/naijarc/prompt_3/naijarc_yor.yaml' 2026-05-11T15:53:28,383 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc' 2026-05-11T15:53:28,384 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_hau.yaml' 2026-05-11T15:53:28,386 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_ibo.yaml' 2026-05-11T15:53:28,387 adding 'lm_eval/tasks/afrobench/naijarc/prompt_4/naijarc_yor.yaml' 2026-05-11T15:53:28,389 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc' 2026-05-11T15:53:28,390 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_hau.yaml' 2026-05-11T15:53:28,391 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_ibo.yaml' 2026-05-11T15:53:28,392 adding 'lm_eval/tasks/afrobench/naijarc/prompt_5/naijarc_yor.yaml' 2026-05-11T15:53:28,394 adding 'lm_eval/tasks/afrobench/nollysenti/README.md' 2026-05-11T15:53:28,395 adding 'lm_eval/tasks/afrobench/nollysenti/nollysenti.yaml' 2026-05-11T15:53:28,397 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti' 2026-05-11T15:53:28,398 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_eng.yaml' 2026-05-11T15:53:28,399 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_hau.yaml' 2026-05-11T15:53:28,400 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_ibo.yaml' 2026-05-11T15:53:28,401 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_pcm.yaml' 2026-05-11T15:53:28,402 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/nollysenti_yor.yaml' 2026-05-11T15:53:28,404 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_1/utils.py' 2026-05-11T15:53:28,406 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti' 2026-05-11T15:53:28,407 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_eng.yaml' 2026-05-11T15:53:28,408 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_hau.yaml' 2026-05-11T15:53:28,409 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_ibo.yaml' 2026-05-11T15:53:28,410 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_pcm.yaml' 2026-05-11T15:53:28,411 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/nollysenti_yor.yaml' 2026-05-11T15:53:28,412 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_2/utils.py' 2026-05-11T15:53:28,414 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti' 2026-05-11T15:53:28,415 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_eng.yaml' 2026-05-11T15:53:28,417 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_hau.yaml' 2026-05-11T15:53:28,418 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_ibo.yaml' 2026-05-11T15:53:28,419 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_pcm.yaml' 2026-05-11T15:53:28,420 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/nollysenti_yor.yaml' 2026-05-11T15:53:28,421 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_3/utils.py' 2026-05-11T15:53:28,423 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti' 2026-05-11T15:53:28,424 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_eng.yaml' 2026-05-11T15:53:28,426 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_hau.yaml' 2026-05-11T15:53:28,427 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_ibo.yaml' 2026-05-11T15:53:28,428 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_pcm.yaml' 2026-05-11T15:53:28,429 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/nollysenti_yor.yaml' 2026-05-11T15:53:28,430 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_4/utils.py' 2026-05-11T15:53:28,432 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti' 2026-05-11T15:53:28,433 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_eng.yaml' 2026-05-11T15:53:28,435 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_hau.yaml' 2026-05-11T15:53:28,436 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_ibo.yaml' 2026-05-11T15:53:28,437 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_pcm.yaml' 2026-05-11T15:53:28,439 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/nollysenti_yor.yaml' 2026-05-11T15:53:28,440 adding 'lm_eval/tasks/afrobench/nollysenti/prompt_5/utils.py' 2026-05-11T15:53:28,442 adding 'lm_eval/tasks/afrobench/ntrex/README.md' 2026-05-11T15:53:28,444 adding 'lm_eval/tasks/afrobench/ntrex/gen_utils.py' 2026-05-11T15:53:28,445 adding 'lm_eval/tasks/afrobench/ntrex/ntrex.yaml' 2026-05-11T15:53:28,448 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex' 2026-05-11T15:53:28,449 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_afr_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,450 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_amh_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,451 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_arb_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,452 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_bem_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,454 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ewe_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,455 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_fra_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,456 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_hau_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,457 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ibo_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,459 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_kin_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,460 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mey_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,461 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_mlg_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,462 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_msa_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,464 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nde_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,465 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nso_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,466 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_nya_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,467 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_orm_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,469 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_shi_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,470 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_sna_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,471 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_som_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,472 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ssw_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,473 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_swa_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,475 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tam_Taml-eng_Latn.yaml' 2026-05-11T15:53:28,476 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tel_Telu-eng_Latn.yaml' 2026-05-11T15:53:28,477 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tir_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,478 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ton_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,479 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_tsn_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,481 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_urd_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,482 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_ven_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,483 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_wol_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,484 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_xho_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,485 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_yor_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,487 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/african-english/ntrex_zul_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,489 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex' 2026-05-11T15:53:28,490 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-afr_Latn.yaml' 2026-05-11T15:53:28,492 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-amh_Ethi.yaml' 2026-05-11T15:53:28,493 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-arb_Arab.yaml' 2026-05-11T15:53:28,494 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-bem_Latn.yaml' 2026-05-11T15:53:28,495 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ewe_Latn.yaml' 2026-05-11T15:53:28,497 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-fra_Latn.yaml' 2026-05-11T15:53:28,498 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-hau_Latn.yaml' 2026-05-11T15:53:28,499 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ibo_Latn.yaml' 2026-05-11T15:53:28,500 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-kin_Latn.yaml' 2026-05-11T15:53:28,501 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mey_Arab.yaml' 2026-05-11T15:53:28,503 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-mlg_Latn.yaml' 2026-05-11T15:53:28,504 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-msa_Latn.yaml' 2026-05-11T15:53:28,505 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nde_Latn.yaml' 2026-05-11T15:53:28,506 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nso_Latn.yaml' 2026-05-11T15:53:28,507 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-nya_Latn.yaml' 2026-05-11T15:53:28,509 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-orm_Ethi.yaml' 2026-05-11T15:53:28,510 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-shi_Arab.yaml' 2026-05-11T15:53:28,511 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-sna_Latn.yaml' 2026-05-11T15:53:28,512 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-som_Latn.yaml' 2026-05-11T15:53:28,513 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ssw_Latn.yaml' 2026-05-11T15:53:28,515 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-swa_Latn.yaml' 2026-05-11T15:53:28,516 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tam_Taml.yaml' 2026-05-11T15:53:28,517 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tel_Telu.yaml' 2026-05-11T15:53:28,518 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tir_Ethi.yaml' 2026-05-11T15:53:28,519 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ton_Latn.yaml' 2026-05-11T15:53:28,521 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-tsn_Latn.yaml' 2026-05-11T15:53:28,522 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-urd_Arab.yaml' 2026-05-11T15:53:28,523 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-ven_Latn.yaml' 2026-05-11T15:53:28,524 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-wol_Latn.yaml' 2026-05-11T15:53:28,526 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-xho_Latn.yaml' 2026-05-11T15:53:28,527 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-yor_Latn.yaml' 2026-05-11T15:53:28,528 adding 'lm_eval/tasks/afrobench/ntrex/prompt_1/english-african/ntrex_eng_Latn-zul_Latn.yaml' 2026-05-11T15:53:28,531 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex' 2026-05-11T15:53:28,532 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_afr_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,533 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_amh_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,535 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_arb_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,536 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_bem_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,537 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ewe_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,538 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_fra_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,539 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_hau_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,540 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ibo_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,541 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_kin_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,542 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mey_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,543 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_mlg_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,545 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_msa_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,546 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nde_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,547 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nso_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,548 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_nya_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,549 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_orm_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,551 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_shi_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,552 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_sna_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,553 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_som_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,554 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ssw_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,555 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_swa_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,556 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tam_Taml-eng_Latn.yaml' 2026-05-11T15:53:28,558 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tel_Telu-eng_Latn.yaml' 2026-05-11T15:53:28,559 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tir_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,560 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ton_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,561 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_tsn_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,562 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_urd_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,564 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_ven_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,565 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_wol_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,566 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_xho_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,567 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_yor_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,569 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/african-english/ntrex_zul_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,571 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex' 2026-05-11T15:53:28,572 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-afr_Latn.yaml' 2026-05-11T15:53:28,573 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-amh_Ethi.yaml' 2026-05-11T15:53:28,575 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-arb_Arab.yaml' 2026-05-11T15:53:28,576 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-bem_Latn.yaml' 2026-05-11T15:53:28,577 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ewe_Latn.yaml' 2026-05-11T15:53:28,578 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-fra_Latn.yaml' 2026-05-11T15:53:28,579 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-hau_Latn.yaml' 2026-05-11T15:53:28,580 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ibo_Latn.yaml' 2026-05-11T15:53:28,581 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-kin_Latn.yaml' 2026-05-11T15:53:28,583 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mey_Arab.yaml' 2026-05-11T15:53:28,584 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-mlg_Latn.yaml' 2026-05-11T15:53:28,585 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-msa_Latn.yaml' 2026-05-11T15:53:28,586 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nde_Latn.yaml' 2026-05-11T15:53:28,587 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nso_Latn.yaml' 2026-05-11T15:53:28,589 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-nya_Latn.yaml' 2026-05-11T15:53:28,590 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-orm_Ethi.yaml' 2026-05-11T15:53:28,591 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-shi_Arab.yaml' 2026-05-11T15:53:28,593 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-sna_Latn.yaml' 2026-05-11T15:53:28,594 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-som_Latn.yaml' 2026-05-11T15:53:28,595 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ssw_Latn.yaml' 2026-05-11T15:53:28,596 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-swa_Latn.yaml' 2026-05-11T15:53:28,597 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tam_Taml.yaml' 2026-05-11T15:53:28,599 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tel_Telu.yaml' 2026-05-11T15:53:28,600 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tir_Ethi.yaml' 2026-05-11T15:53:28,601 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ton_Latn.yaml' 2026-05-11T15:53:28,602 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-tsn_Latn.yaml' 2026-05-11T15:53:28,603 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-urd_Arab.yaml' 2026-05-11T15:53:28,605 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-ven_Latn.yaml' 2026-05-11T15:53:28,606 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-wol_Latn.yaml' 2026-05-11T15:53:28,607 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-xho_Latn.yaml' 2026-05-11T15:53:28,608 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-yor_Latn.yaml' 2026-05-11T15:53:28,610 adding 'lm_eval/tasks/afrobench/ntrex/prompt_2/english-african/ntrex_eng_Latn-zul_Latn.yaml' 2026-05-11T15:53:28,612 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex' 2026-05-11T15:53:28,613 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_afr_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,615 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_amh_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,616 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_arb_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,617 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_bem_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,618 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ewe_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,619 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_fra_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,621 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_hau_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,622 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ibo_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,623 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_kin_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,624 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mey_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,625 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_mlg_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,626 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_msa_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,628 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nde_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,629 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nso_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,630 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_nya_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,631 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_orm_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,633 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_shi_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,634 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_sna_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,635 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_som_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,636 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ssw_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,637 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_swa_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,639 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tam_Taml-eng_Latn.yaml' 2026-05-11T15:53:28,640 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tel_Telu-eng_Latn.yaml' 2026-05-11T15:53:28,641 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tir_Ethi-eng_Latn.yaml' 2026-05-11T15:53:28,642 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ton_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,644 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_tsn_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,645 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_urd_Arab-eng_Latn.yaml' 2026-05-11T15:53:28,646 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_ven_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,647 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_wol_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,649 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_xho_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,650 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_yor_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,651 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/african-english/ntrex_zul_Latn-eng_Latn.yaml' 2026-05-11T15:53:28,653 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex' 2026-05-11T15:53:28,655 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-afr_Latn.yaml' 2026-05-11T15:53:28,656 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-amh_Ethi.yaml' 2026-05-11T15:53:28,657 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-arb_Arab.yaml' 2026-05-11T15:53:28,658 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-bem_Latn.yaml' 2026-05-11T15:53:28,660 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ewe_Latn.yaml' 2026-05-11T15:53:28,661 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-fra_Latn.yaml' 2026-05-11T15:53:28,662 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-hau_Latn.yaml' 2026-05-11T15:53:28,664 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ibo_Latn.yaml' 2026-05-11T15:53:28,665 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-kin_Latn.yaml' 2026-05-11T15:53:28,666 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mey_Arab.yaml' 2026-05-11T15:53:28,667 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-mlg_Latn.yaml' 2026-05-11T15:53:28,672 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-msa_Latn.yaml' 2026-05-11T15:53:28,673 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nde_Latn.yaml' 2026-05-11T15:53:28,674 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nso_Latn.yaml' 2026-05-11T15:53:28,675 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-nya_Latn.yaml' 2026-05-11T15:53:28,676 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-orm_Ethi.yaml' 2026-05-11T15:53:28,677 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-shi_Arab.yaml' 2026-05-11T15:53:28,679 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-sna_Latn.yaml' 2026-05-11T15:53:28,680 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-som_Latn.yaml' 2026-05-11T15:53:28,681 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ssw_Latn.yaml' 2026-05-11T15:53:28,682 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-swa_Latn.yaml' 2026-05-11T15:53:28,683 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tam_Taml.yaml' 2026-05-11T15:53:28,685 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tel_Telu.yaml' 2026-05-11T15:53:28,686 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tir_Ethi.yaml' 2026-05-11T15:53:28,687 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ton_Latn.yaml' 2026-05-11T15:53:28,688 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-tsn_Latn.yaml' 2026-05-11T15:53:28,689 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-urd_Arab.yaml' 2026-05-11T15:53:28,691 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-ven_Latn.yaml' 2026-05-11T15:53:28,692 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-wol_Latn.yaml' 2026-05-11T15:53:28,693 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-xho_Latn.yaml' 2026-05-11T15:53:28,694 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-yor_Latn.yaml' 2026-05-11T15:53:28,696 adding 'lm_eval/tasks/afrobench/ntrex/prompt_3/english-african/ntrex_eng_Latn-zul_Latn.yaml' 2026-05-11T15:53:28,698 adding 'lm_eval/tasks/afrobench/openai_mmlu/README.md' 2026-05-11T15:53:28,699 adding 'lm_eval/tasks/afrobench/openai_mmlu/openai_mmlu.yaml' 2026-05-11T15:53:28,700 adding 'lm_eval/tasks/afrobench/openai_mmlu/utils.py' 2026-05-11T15:53:28,702 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu' 2026-05-11T15:53:28,703 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_ara.yaml' 2026-05-11T15:53:28,705 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_swa.yaml' 2026-05-11T15:53:28,706 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_1/openai_mmlu_yor.yaml' 2026-05-11T15:53:28,708 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu' 2026-05-11T15:53:28,710 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_ara.yaml' 2026-05-11T15:53:28,711 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_swa.yaml' 2026-05-11T15:53:28,712 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_2/openai_mmlu_yor.yaml' 2026-05-11T15:53:28,714 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu' 2026-05-11T15:53:28,715 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_ara.yaml' 2026-05-11T15:53:28,716 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_swa.yaml' 2026-05-11T15:53:28,717 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_3/openai_mmlu_yor.yaml' 2026-05-11T15:53:28,719 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu' 2026-05-11T15:53:28,720 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_ara.yaml' 2026-05-11T15:53:28,721 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_swa.yaml' 2026-05-11T15:53:28,722 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_4/openai_mmlu_yor.yaml' 2026-05-11T15:53:28,724 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu' 2026-05-11T15:53:28,725 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_ara.yaml' 2026-05-11T15:53:28,726 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_swa.yaml' 2026-05-11T15:53:28,728 adding 'lm_eval/tasks/afrobench/openai_mmlu/prompt_5/openai_mmlu_yor.yaml' 2026-05-11T15:53:28,729 adding 'lm_eval/tasks/afrobench/salt/README.md' 2026-05-11T15:53:28,731 adding 'lm_eval/tasks/afrobench/salt/gen_utils.py' 2026-05-11T15:53:28,732 adding 'lm_eval/tasks/afrobench/salt/salt.yaml' 2026-05-11T15:53:28,734 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt' 2026-05-11T15:53:28,735 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_ach-eng.yaml' 2026-05-11T15:53:28,736 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ach.yaml' 2026-05-11T15:53:28,737 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-ibo.yaml' 2026-05-11T15:53:28,739 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lgg.yaml' 2026-05-11T15:53:28,740 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-lug.yaml' 2026-05-11T15:53:28,741 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-nyn.yaml' 2026-05-11T15:53:28,742 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-swa.yaml' 2026-05-11T15:53:28,743 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_eng-teo.yaml' 2026-05-11T15:53:28,744 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_ibo-eng.yaml' 2026-05-11T15:53:28,746 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_lgg-eng.yaml' 2026-05-11T15:53:28,747 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_lug-eng.yaml' 2026-05-11T15:53:28,748 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_nyn-eng.yaml' 2026-05-11T15:53:28,749 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_swa-eng.yaml' 2026-05-11T15:53:28,750 adding 'lm_eval/tasks/afrobench/salt/prompt_1/salt_teo-eng.yaml' 2026-05-11T15:53:28,752 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt' 2026-05-11T15:53:28,753 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_ach-eng.yaml' 2026-05-11T15:53:28,755 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ach.yaml' 2026-05-11T15:53:28,756 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-ibo.yaml' 2026-05-11T15:53:28,757 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lgg.yaml' 2026-05-11T15:53:28,759 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-lug.yaml' 2026-05-11T15:53:28,760 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-nyn.yaml' 2026-05-11T15:53:28,761 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-swa.yaml' 2026-05-11T15:53:28,762 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_eng-teo.yaml' 2026-05-11T15:53:28,763 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_ibo-eng.yaml' 2026-05-11T15:53:28,764 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_lgg-eng.yaml' 2026-05-11T15:53:28,766 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_lug-eng.yaml' 2026-05-11T15:53:28,767 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_nyn-eng.yaml' 2026-05-11T15:53:28,768 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_swa-eng.yaml' 2026-05-11T15:53:28,769 adding 'lm_eval/tasks/afrobench/salt/prompt_2/salt_teo-eng.yaml' 2026-05-11T15:53:28,771 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt' 2026-05-11T15:53:28,773 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_ach-eng.yaml' 2026-05-11T15:53:28,774 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ach.yaml' 2026-05-11T15:53:28,775 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-ibo.yaml' 2026-05-11T15:53:28,776 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lgg.yaml' 2026-05-11T15:53:28,777 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-lug.yaml' 2026-05-11T15:53:28,778 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-nyn.yaml' 2026-05-11T15:53:28,779 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-swa.yaml' 2026-05-11T15:53:28,780 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_eng-teo.yaml' 2026-05-11T15:53:28,782 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_ibo-eng.yaml' 2026-05-11T15:53:28,783 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_lgg-eng.yaml' 2026-05-11T15:53:28,784 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_lug-eng.yaml' 2026-05-11T15:53:28,785 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_nyn-eng.yaml' 2026-05-11T15:53:28,786 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_swa-eng.yaml' 2026-05-11T15:53:28,788 adding 'lm_eval/tasks/afrobench/salt/prompt_3/salt_teo-eng.yaml' 2026-05-11T15:53:28,789 adding 'lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench.sh' 2026-05-11T15:53:28,791 adding 'lm_eval/tasks/afrobench/sample_run_scripts/run_afrobench_lite.sh' 2026-05-11T15:53:28,792 adding 'lm_eval/tasks/afrobench/sib/README.md' 2026-05-11T15:53:28,794 adding 'lm_eval/tasks/afrobench/sib/sib.yaml' 2026-05-11T15:53:28,795 adding 'lm_eval/tasks/afrobench/sib/utils.py' 2026-05-11T15:53:28,798 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib' 2026-05-11T15:53:28,799 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_aeb.yaml' 2026-05-11T15:53:28,800 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_afr.yaml' 2026-05-11T15:53:28,802 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_aka.yaml' 2026-05-11T15:53:28,803 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_amh.yaml' 2026-05-11T15:53:28,804 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ary.yaml' 2026-05-11T15:53:28,805 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_arz.yaml' 2026-05-11T15:53:28,806 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_bam.yaml' 2026-05-11T15:53:28,808 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_bem.yaml' 2026-05-11T15:53:28,809 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_cjk.yaml' 2026-05-11T15:53:28,810 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_dik.yaml' 2026-05-11T15:53:28,811 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_dyu.yaml' 2026-05-11T15:53:28,812 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_eng.yaml' 2026-05-11T15:53:28,814 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ewe.yaml' 2026-05-11T15:53:28,815 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_fon.yaml' 2026-05-11T15:53:28,816 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_fra.yaml' 2026-05-11T15:53:28,817 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_fuv.yaml' 2026-05-11T15:53:28,819 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_gaz.yaml' 2026-05-11T15:53:28,820 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_hau.yaml' 2026-05-11T15:53:28,821 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ibo.yaml' 2026-05-11T15:53:28,822 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kab.yaml' 2026-05-11T15:53:28,823 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kam.yaml' 2026-05-11T15:53:28,825 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kbp.yaml' 2026-05-11T15:53:28,826 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kea.yaml' 2026-05-11T15:53:28,827 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kik.yaml' 2026-05-11T15:53:28,829 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kin.yaml' 2026-05-11T15:53:28,830 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kmb.yaml' 2026-05-11T15:53:28,831 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_knc.yaml' 2026-05-11T15:53:28,832 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_kon.yaml' 2026-05-11T15:53:28,833 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_lin.yaml' 2026-05-11T15:53:28,834 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_lua.yaml' 2026-05-11T15:53:28,835 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_lug.yaml' 2026-05-11T15:53:28,836 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_luo.yaml' 2026-05-11T15:53:28,838 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_mos.yaml' 2026-05-11T15:53:28,839 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_nso.yaml' 2026-05-11T15:53:28,840 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_nus.yaml' 2026-05-11T15:53:28,841 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_nya.yaml' 2026-05-11T15:53:28,842 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_plt.yaml' 2026-05-11T15:53:28,843 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_por.yaml' 2026-05-11T15:53:28,844 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_run.yaml' 2026-05-11T15:53:28,845 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_sag.yaml' 2026-05-11T15:53:28,847 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_sna.yaml' 2026-05-11T15:53:28,848 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_som.yaml' 2026-05-11T15:53:28,850 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_sot.yaml' 2026-05-11T15:53:28,851 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_ssw.yaml' 2026-05-11T15:53:28,852 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_swa.yaml' 2026-05-11T15:53:28,853 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_taq.yaml' 2026-05-11T15:53:28,854 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tir.yaml' 2026-05-11T15:53:28,855 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tso.yaml' 2026-05-11T15:53:28,857 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tum.yaml' 2026-05-11T15:53:28,858 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_twi.yaml' 2026-05-11T15:53:28,859 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_tzm.yaml' 2026-05-11T15:53:28,860 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_umb.yaml' 2026-05-11T15:53:28,861 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_wol.yaml' 2026-05-11T15:53:28,862 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_xho.yaml' 2026-05-11T15:53:28,864 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_yor.yaml' 2026-05-11T15:53:28,865 adding 'lm_eval/tasks/afrobench/sib/prompt_1/sib_zul.yaml' 2026-05-11T15:53:28,866 adding 'lm_eval/tasks/afrobench/sib/prompt_1/utils.py' 2026-05-11T15:53:28,869 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib' 2026-05-11T15:53:28,870 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_aeb.yaml' 2026-05-11T15:53:28,871 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_afr.yaml' 2026-05-11T15:53:28,872 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_aka.yaml' 2026-05-11T15:53:28,874 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_amh.yaml' 2026-05-11T15:53:28,875 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ary.yaml' 2026-05-11T15:53:28,876 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_arz.yaml' 2026-05-11T15:53:28,877 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_bam.yaml' 2026-05-11T15:53:28,878 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_bem.yaml' 2026-05-11T15:53:28,880 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_cjk.yaml' 2026-05-11T15:53:28,881 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_dik.yaml' 2026-05-11T15:53:28,882 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_dyu.yaml' 2026-05-11T15:53:28,883 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_eng.yaml' 2026-05-11T15:53:28,884 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ewe.yaml' 2026-05-11T15:53:28,886 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_fon.yaml' 2026-05-11T15:53:28,887 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_fra.yaml' 2026-05-11T15:53:28,888 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_fuv.yaml' 2026-05-11T15:53:28,889 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_gaz.yaml' 2026-05-11T15:53:28,891 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_hau.yaml' 2026-05-11T15:53:28,892 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ibo.yaml' 2026-05-11T15:53:28,893 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kab.yaml' 2026-05-11T15:53:28,894 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kam.yaml' 2026-05-11T15:53:28,895 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kbp.yaml' 2026-05-11T15:53:28,896 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kea.yaml' 2026-05-11T15:53:28,897 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kik.yaml' 2026-05-11T15:53:28,898 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kin.yaml' 2026-05-11T15:53:28,899 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kmb.yaml' 2026-05-11T15:53:28,900 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_knc.yaml' 2026-05-11T15:53:28,901 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_kon.yaml' 2026-05-11T15:53:28,903 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_lin.yaml' 2026-05-11T15:53:28,904 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_lua.yaml' 2026-05-11T15:53:28,905 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_lug.yaml' 2026-05-11T15:53:28,906 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_luo.yaml' 2026-05-11T15:53:28,907 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_mos.yaml' 2026-05-11T15:53:28,909 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_nso.yaml' 2026-05-11T15:53:28,910 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_nus.yaml' 2026-05-11T15:53:28,911 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_nya.yaml' 2026-05-11T15:53:28,912 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_plt.yaml' 2026-05-11T15:53:28,913 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_por.yaml' 2026-05-11T15:53:28,914 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_run.yaml' 2026-05-11T15:53:28,915 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_sag.yaml' 2026-05-11T15:53:28,916 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_sna.yaml' 2026-05-11T15:53:28,918 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_som.yaml' 2026-05-11T15:53:28,919 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_sot.yaml' 2026-05-11T15:53:28,920 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_ssw.yaml' 2026-05-11T15:53:28,921 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_swa.yaml' 2026-05-11T15:53:28,922 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_taq.yaml' 2026-05-11T15:53:28,924 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tir.yaml' 2026-05-11T15:53:28,925 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tso.yaml' 2026-05-11T15:53:28,926 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tum.yaml' 2026-05-11T15:53:28,927 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_twi.yaml' 2026-05-11T15:53:28,928 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_tzm.yaml' 2026-05-11T15:53:28,930 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_umb.yaml' 2026-05-11T15:53:28,931 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_wol.yaml' 2026-05-11T15:53:28,932 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_xho.yaml' 2026-05-11T15:53:28,933 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_yor.yaml' 2026-05-11T15:53:28,935 adding 'lm_eval/tasks/afrobench/sib/prompt_2/sib_zul.yaml' 2026-05-11T15:53:28,936 adding 'lm_eval/tasks/afrobench/sib/prompt_2/utils.py' 2026-05-11T15:53:28,939 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib' 2026-05-11T15:53:28,940 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_aeb.yaml' 2026-05-11T15:53:28,941 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_afr.yaml' 2026-05-11T15:53:28,942 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_aka.yaml' 2026-05-11T15:53:28,944 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_amh.yaml' 2026-05-11T15:53:28,945 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ary.yaml' 2026-05-11T15:53:28,946 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_arz.yaml' 2026-05-11T15:53:28,947 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_bam.yaml' 2026-05-11T15:53:28,948 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_bem.yaml' 2026-05-11T15:53:28,950 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_cjk.yaml' 2026-05-11T15:53:28,951 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_dik.yaml' 2026-05-11T15:53:28,952 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_dyu.yaml' 2026-05-11T15:53:28,953 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_eng.yaml' 2026-05-11T15:53:28,954 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ewe.yaml' 2026-05-11T15:53:28,955 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_fon.yaml' 2026-05-11T15:53:28,956 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_fra.yaml' 2026-05-11T15:53:28,957 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_fuv.yaml' 2026-05-11T15:53:28,959 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_gaz.yaml' 2026-05-11T15:53:28,960 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_hau.yaml' 2026-05-11T15:53:28,961 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ibo.yaml' 2026-05-11T15:53:28,962 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kab.yaml' 2026-05-11T15:53:28,963 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kam.yaml' 2026-05-11T15:53:28,965 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kbp.yaml' 2026-05-11T15:53:28,966 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kea.yaml' 2026-05-11T15:53:28,967 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kik.yaml' 2026-05-11T15:53:28,968 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kin.yaml' 2026-05-11T15:53:28,969 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kmb.yaml' 2026-05-11T15:53:28,970 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_knc.yaml' 2026-05-11T15:53:28,972 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_kon.yaml' 2026-05-11T15:53:28,973 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_lin.yaml' 2026-05-11T15:53:28,974 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_lua.yaml' 2026-05-11T15:53:28,975 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_lug.yaml' 2026-05-11T15:53:28,976 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_luo.yaml' 2026-05-11T15:53:28,977 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_mos.yaml' 2026-05-11T15:53:28,979 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_nso.yaml' 2026-05-11T15:53:28,980 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_nus.yaml' 2026-05-11T15:53:28,981 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_nya.yaml' 2026-05-11T15:53:28,982 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_plt.yaml' 2026-05-11T15:53:28,984 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_por.yaml' 2026-05-11T15:53:28,985 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_run.yaml' 2026-05-11T15:53:28,986 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_sag.yaml' 2026-05-11T15:53:28,987 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_sna.yaml' 2026-05-11T15:53:28,988 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_som.yaml' 2026-05-11T15:53:28,990 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_sot.yaml' 2026-05-11T15:53:28,991 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_ssw.yaml' 2026-05-11T15:53:28,992 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_swa.yaml' 2026-05-11T15:53:28,993 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_taq.yaml' 2026-05-11T15:53:28,995 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tir.yaml' 2026-05-11T15:53:28,996 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tso.yaml' 2026-05-11T15:53:28,997 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tum.yaml' 2026-05-11T15:53:28,998 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_twi.yaml' 2026-05-11T15:53:28,999 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_tzm.yaml' 2026-05-11T15:53:29,001 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_umb.yaml' 2026-05-11T15:53:29,002 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_wol.yaml' 2026-05-11T15:53:29,003 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_xho.yaml' 2026-05-11T15:53:29,004 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_yor.yaml' 2026-05-11T15:53:29,006 adding 'lm_eval/tasks/afrobench/sib/prompt_3/sib_zul.yaml' 2026-05-11T15:53:29,007 adding 'lm_eval/tasks/afrobench/sib/prompt_3/utils.py' 2026-05-11T15:53:29,010 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib' 2026-05-11T15:53:29,011 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_aeb.yaml' 2026-05-11T15:53:29,012 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_afr.yaml' 2026-05-11T15:53:29,013 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_aka.yaml' 2026-05-11T15:53:29,014 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_amh.yaml' 2026-05-11T15:53:29,015 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ary.yaml' 2026-05-11T15:53:29,016 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_arz.yaml' 2026-05-11T15:53:29,017 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_bam.yaml' 2026-05-11T15:53:29,019 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_bem.yaml' 2026-05-11T15:53:29,020 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_cjk.yaml' 2026-05-11T15:53:29,021 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_dik.yaml' 2026-05-11T15:53:29,022 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_dyu.yaml' 2026-05-11T15:53:29,023 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_eng.yaml' 2026-05-11T15:53:29,025 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ewe.yaml' 2026-05-11T15:53:29,026 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_fon.yaml' 2026-05-11T15:53:29,027 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_fra.yaml' 2026-05-11T15:53:29,028 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_fuv.yaml' 2026-05-11T15:53:29,029 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_gaz.yaml' 2026-05-11T15:53:29,030 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_hau.yaml' 2026-05-11T15:53:29,032 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ibo.yaml' 2026-05-11T15:53:29,033 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kab.yaml' 2026-05-11T15:53:29,034 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kam.yaml' 2026-05-11T15:53:29,035 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kbp.yaml' 2026-05-11T15:53:29,036 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kea.yaml' 2026-05-11T15:53:29,037 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kik.yaml' 2026-05-11T15:53:29,039 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kin.yaml' 2026-05-11T15:53:29,040 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kmb.yaml' 2026-05-11T15:53:29,041 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_knc.yaml' 2026-05-11T15:53:29,042 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_kon.yaml' 2026-05-11T15:53:29,043 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_lin.yaml' 2026-05-11T15:53:29,045 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_lua.yaml' 2026-05-11T15:53:29,046 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_lug.yaml' 2026-05-11T15:53:29,047 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_luo.yaml' 2026-05-11T15:53:29,049 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_mos.yaml' 2026-05-11T15:53:29,050 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_nso.yaml' 2026-05-11T15:53:29,051 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_nus.yaml' 2026-05-11T15:53:29,052 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_nya.yaml' 2026-05-11T15:53:29,053 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_plt.yaml' 2026-05-11T15:53:29,055 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_por.yaml' 2026-05-11T15:53:29,056 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_run.yaml' 2026-05-11T15:53:29,057 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_sag.yaml' 2026-05-11T15:53:29,059 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_sna.yaml' 2026-05-11T15:53:29,060 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_som.yaml' 2026-05-11T15:53:29,061 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_sot.yaml' 2026-05-11T15:53:29,063 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_ssw.yaml' 2026-05-11T15:53:29,064 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_swa.yaml' 2026-05-11T15:53:29,065 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_taq.yaml' 2026-05-11T15:53:29,066 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tir.yaml' 2026-05-11T15:53:29,067 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tso.yaml' 2026-05-11T15:53:29,068 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tum.yaml' 2026-05-11T15:53:29,070 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_twi.yaml' 2026-05-11T15:53:29,071 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_tzm.yaml' 2026-05-11T15:53:29,072 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_umb.yaml' 2026-05-11T15:53:29,073 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_wol.yaml' 2026-05-11T15:53:29,074 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_xho.yaml' 2026-05-11T15:53:29,075 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_yor.yaml' 2026-05-11T15:53:29,076 adding 'lm_eval/tasks/afrobench/sib/prompt_4/sib_zul.yaml' 2026-05-11T15:53:29,077 adding 'lm_eval/tasks/afrobench/sib/prompt_4/utils.py' 2026-05-11T15:53:29,080 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib' 2026-05-11T15:53:29,081 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_aeb.yaml' 2026-05-11T15:53:29,082 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_afr.yaml' 2026-05-11T15:53:29,084 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_aka.yaml' 2026-05-11T15:53:29,085 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_amh.yaml' 2026-05-11T15:53:29,086 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ary.yaml' 2026-05-11T15:53:29,087 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_arz.yaml' 2026-05-11T15:53:29,089 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_bam.yaml' 2026-05-11T15:53:29,090 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_bem.yaml' 2026-05-11T15:53:29,091 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_cjk.yaml' 2026-05-11T15:53:29,093 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_dik.yaml' 2026-05-11T15:53:29,094 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_dyu.yaml' 2026-05-11T15:53:29,095 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_eng.yaml' 2026-05-11T15:53:29,096 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ewe.yaml' 2026-05-11T15:53:29,098 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_fon.yaml' 2026-05-11T15:53:29,099 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_fra.yaml' 2026-05-11T15:53:29,100 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_fuv.yaml' 2026-05-11T15:53:29,101 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_gaz.yaml' 2026-05-11T15:53:29,103 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_hau.yaml' 2026-05-11T15:53:29,104 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ibo.yaml' 2026-05-11T15:53:29,105 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kab.yaml' 2026-05-11T15:53:29,107 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kam.yaml' 2026-05-11T15:53:29,108 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kbp.yaml' 2026-05-11T15:53:29,109 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kea.yaml' 2026-05-11T15:53:29,111 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kik.yaml' 2026-05-11T15:53:29,112 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kin.yaml' 2026-05-11T15:53:29,113 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kmb.yaml' 2026-05-11T15:53:29,114 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_knc.yaml' 2026-05-11T15:53:29,116 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_kon.yaml' 2026-05-11T15:53:29,117 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_lin.yaml' 2026-05-11T15:53:29,118 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_lua.yaml' 2026-05-11T15:53:29,120 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_lug.yaml' 2026-05-11T15:53:29,121 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_luo.yaml' 2026-05-11T15:53:29,122 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_mos.yaml' 2026-05-11T15:53:29,123 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_nso.yaml' 2026-05-11T15:53:29,125 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_nus.yaml' 2026-05-11T15:53:29,126 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_nya.yaml' 2026-05-11T15:53:29,127 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_plt.yaml' 2026-05-11T15:53:29,129 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_por.yaml' 2026-05-11T15:53:29,130 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_run.yaml' 2026-05-11T15:53:29,131 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_sag.yaml' 2026-05-11T15:53:29,132 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_sna.yaml' 2026-05-11T15:53:29,133 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_som.yaml' 2026-05-11T15:53:29,134 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_sot.yaml' 2026-05-11T15:53:29,136 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_ssw.yaml' 2026-05-11T15:53:29,137 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_swa.yaml' 2026-05-11T15:53:29,138 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_taq.yaml' 2026-05-11T15:53:29,139 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tir.yaml' 2026-05-11T15:53:29,141 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tso.yaml' 2026-05-11T15:53:29,142 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tum.yaml' 2026-05-11T15:53:29,143 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_twi.yaml' 2026-05-11T15:53:29,144 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_tzm.yaml' 2026-05-11T15:53:29,145 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_umb.yaml' 2026-05-11T15:53:29,147 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_wol.yaml' 2026-05-11T15:53:29,148 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_xho.yaml' 2026-05-11T15:53:29,149 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_yor.yaml' 2026-05-11T15:53:29,150 adding 'lm_eval/tasks/afrobench/sib/prompt_5/sib_zul.yaml' 2026-05-11T15:53:29,151 adding 'lm_eval/tasks/afrobench/sib/prompt_5/utils.py' 2026-05-11T15:53:29,153 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/README.md' 2026-05-11T15:53:29,154 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/uhura.yaml' 2026-05-11T15:53:29,156 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/utils.py' 2026-05-11T15:53:29,158 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy' 2026-05-11T15:53:29,159 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_am.yaml' 2026-05-11T15:53:29,160 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_en.yaml' 2026-05-11T15:53:29,161 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_ha.yaml' 2026-05-11T15:53:29,163 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_nso.yaml' 2026-05-11T15:53:29,164 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_sw.yaml' 2026-05-11T15:53:29,165 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_yo.yaml' 2026-05-11T15:53:29,166 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/uhura-arc-easy_zu.yaml' 2026-05-11T15:53:29,167 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_1/utils.py' 2026-05-11T15:53:29,169 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy' 2026-05-11T15:53:29,171 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_am.yaml' 2026-05-11T15:53:29,172 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_en.yaml' 2026-05-11T15:53:29,173 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_ha.yaml' 2026-05-11T15:53:29,174 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_nso.yaml' 2026-05-11T15:53:29,175 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_sw.yaml' 2026-05-11T15:53:29,177 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_yo.yaml' 2026-05-11T15:53:29,178 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/uhura-arc-easy_zu.yaml' 2026-05-11T15:53:29,179 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_2/utils.py' 2026-05-11T15:53:29,181 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy' 2026-05-11T15:53:29,182 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_am.yaml' 2026-05-11T15:53:29,183 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_en.yaml' 2026-05-11T15:53:29,185 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_ha.yaml' 2026-05-11T15:53:29,186 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_nso.yaml' 2026-05-11T15:53:29,187 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_sw.yaml' 2026-05-11T15:53:29,188 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_yo.yaml' 2026-05-11T15:53:29,189 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/uhura-arc-easy_zu.yaml' 2026-05-11T15:53:29,190 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_3/utils.py' 2026-05-11T15:53:29,192 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy' 2026-05-11T15:53:29,193 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_am.yaml' 2026-05-11T15:53:29,194 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_en.yaml' 2026-05-11T15:53:29,196 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_ha.yaml' 2026-05-11T15:53:29,197 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_nso.yaml' 2026-05-11T15:53:29,198 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_sw.yaml' 2026-05-11T15:53:29,199 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_yo.yaml' 2026-05-11T15:53:29,200 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/uhura-arc-easy_zu.yaml' 2026-05-11T15:53:29,201 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_4/utils.py' 2026-05-11T15:53:29,203 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy' 2026-05-11T15:53:29,204 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_am.yaml' 2026-05-11T15:53:29,206 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_en.yaml' 2026-05-11T15:53:29,207 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_ha.yaml' 2026-05-11T15:53:29,208 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_nso.yaml' 2026-05-11T15:53:29,209 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_sw.yaml' 2026-05-11T15:53:29,211 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_yo.yaml' 2026-05-11T15:53:29,212 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/uhura-arc-easy_zu.yaml' 2026-05-11T15:53:29,213 adding 'lm_eval/tasks/afrobench/uhura-arc-easy/prompt_5/utils.py' 2026-05-11T15:53:29,215 adding 'lm_eval/tasks/afrobench/xlsum/README.md' 2026-05-11T15:53:29,216 adding 'lm_eval/tasks/afrobench/xlsum/utils.py' 2026-05-11T15:53:29,217 adding 'lm_eval/tasks/afrobench/xlsum/xlsum.yaml' 2026-05-11T15:53:29,219 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/utils.py' 2026-05-11T15:53:29,221 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum' 2026-05-11T15:53:29,222 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_amharic.yaml' 2026-05-11T15:53:29,223 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_arabic.yaml' 2026-05-11T15:53:29,224 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_hausa.yaml' 2026-05-11T15:53:29,226 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_igbo.yaml' 2026-05-11T15:53:29,227 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_kirundi.yaml' 2026-05-11T15:53:29,228 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_oromo.yaml' 2026-05-11T15:53:29,229 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_pidgin.yaml' 2026-05-11T15:53:29,231 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_somali.yaml' 2026-05-11T15:53:29,232 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_swahili.yaml' 2026-05-11T15:53:29,233 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_telugu.yaml' 2026-05-11T15:53:29,234 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_tigrinya.yaml' 2026-05-11T15:53:29,236 adding 'lm_eval/tasks/afrobench/xlsum/prompt_1/xlsum_yoruba.yaml' 2026-05-11T15:53:29,237 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/utils.py' 2026-05-11T15:53:29,239 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum' 2026-05-11T15:53:29,240 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_amharic.yaml' 2026-05-11T15:53:29,241 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_arabic.yaml' 2026-05-11T15:53:29,242 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_hausa.yaml' 2026-05-11T15:53:29,243 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_igbo.yaml' 2026-05-11T15:53:29,244 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_kirundi.yaml' 2026-05-11T15:53:29,246 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_oromo.yaml' 2026-05-11T15:53:29,247 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_pidgin.yaml' 2026-05-11T15:53:29,248 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_somali.yaml' 2026-05-11T15:53:29,249 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_swahili.yaml' 2026-05-11T15:53:29,250 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_telugu.yaml' 2026-05-11T15:53:29,251 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_tigrinya.yaml' 2026-05-11T15:53:29,253 adding 'lm_eval/tasks/afrobench/xlsum/prompt_2/xlsum_yoruba.yaml' 2026-05-11T15:53:29,254 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/utils.py' 2026-05-11T15:53:29,256 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum' 2026-05-11T15:53:29,257 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_amharic.yaml' 2026-05-11T15:53:29,258 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_arabic.yaml' 2026-05-11T15:53:29,259 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_hausa.yaml' 2026-05-11T15:53:29,260 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_igbo.yaml' 2026-05-11T15:53:29,261 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_kirundi.yaml' 2026-05-11T15:53:29,263 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_oromo.yaml' 2026-05-11T15:53:29,264 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_pidgin.yaml' 2026-05-11T15:53:29,265 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_somali.yaml' 2026-05-11T15:53:29,266 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_swahili.yaml' 2026-05-11T15:53:29,267 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_telugu.yaml' 2026-05-11T15:53:29,269 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_tigrinya.yaml' 2026-05-11T15:53:29,270 adding 'lm_eval/tasks/afrobench/xlsum/prompt_3/xlsum_yoruba.yaml' 2026-05-11T15:53:29,273 adding 'lm_eval/tasks/agieval/README.md' 2026-05-11T15:53:29,274 adding 'lm_eval/tasks/agieval/agieval.yaml' 2026-05-11T15:53:29,275 adding 'lm_eval/tasks/agieval/agieval_cn.yaml' 2026-05-11T15:53:29,276 adding 'lm_eval/tasks/agieval/agieval_en.yaml' 2026-05-11T15:53:29,278 adding 'lm_eval/tasks/agieval/agieval_nous.yaml' 2026-05-11T15:53:29,279 adding 'lm_eval/tasks/agieval/aqua-rat.yaml' 2026-05-11T15:53:29,280 adding 'lm_eval/tasks/agieval/gaokao-biology.yaml' 2026-05-11T15:53:29,281 adding 'lm_eval/tasks/agieval/gaokao-chemistry.yaml' 2026-05-11T15:53:29,283 adding 'lm_eval/tasks/agieval/gaokao-chinese.yaml' 2026-05-11T15:53:29,284 adding 'lm_eval/tasks/agieval/gaokao-english.yaml' 2026-05-11T15:53:29,285 adding 'lm_eval/tasks/agieval/gaokao-geography.yaml' 2026-05-11T15:53:29,286 adding 'lm_eval/tasks/agieval/gaokao-history.yaml' 2026-05-11T15:53:29,287 adding 'lm_eval/tasks/agieval/gaokao-mathcloze.yaml' 2026-05-11T15:53:29,288 adding 'lm_eval/tasks/agieval/gaokao-mathqa.yaml' 2026-05-11T15:53:29,290 adding 'lm_eval/tasks/agieval/gaokao-physics.yaml' 2026-05-11T15:53:29,291 adding 'lm_eval/tasks/agieval/jec-qa-ca.yaml' 2026-05-11T15:53:29,292 adding 'lm_eval/tasks/agieval/jec-qa-kd.yaml' 2026-05-11T15:53:29,293 adding 'lm_eval/tasks/agieval/logiqa-en.yaml' 2026-05-11T15:53:29,294 adding 'lm_eval/tasks/agieval/logiqa-zh.yaml' 2026-05-11T15:53:29,295 adding 'lm_eval/tasks/agieval/lsat-ar.yaml' 2026-05-11T15:53:29,296 adding 'lm_eval/tasks/agieval/lsat-lr.yaml' 2026-05-11T15:53:29,297 adding 'lm_eval/tasks/agieval/lsat-rc.yaml' 2026-05-11T15:53:29,299 adding 'lm_eval/tasks/agieval/math.yaml' 2026-05-11T15:53:29,300 adding 'lm_eval/tasks/agieval/sat-en-without-passage.yaml' 2026-05-11T15:53:29,301 adding 'lm_eval/tasks/agieval/sat-en.yaml' 2026-05-11T15:53:29,302 adding 'lm_eval/tasks/agieval/sat-math.yaml' 2026-05-11T15:53:29,304 adding 'lm_eval/tasks/agieval/utils.py' 2026-05-11T15:53:29,306 adding 'lm_eval/tasks/aime/README.md' 2026-05-11T15:53:29,307 adding 'lm_eval/tasks/aime/aime.yaml' 2026-05-11T15:53:29,308 adding 'lm_eval/tasks/aime/aime24.yaml' 2026-05-11T15:53:29,309 adding 'lm_eval/tasks/aime/aime25.yaml' 2026-05-11T15:53:29,311 adding 'lm_eval/tasks/aime/utils.py' 2026-05-11T15:53:29,313 adding 'lm_eval/tasks/alghafa/copa_ar/README.md' 2026-05-11T15:53:29,314 adding 'lm_eval/tasks/alghafa/copa_ar/copa_ar.yaml' 2026-05-11T15:53:29,316 adding 'lm_eval/tasks/alghafa/piqa_ar/README.md' 2026-05-11T15:53:29,317 adding 'lm_eval/tasks/alghafa/piqa_ar/piqa_ar.yaml' 2026-05-11T15:53:29,319 adding 'lm_eval/tasks/anli/README.md' 2026-05-11T15:53:29,320 adding 'lm_eval/tasks/anli/anli_r1.yaml' 2026-05-11T15:53:29,321 adding 'lm_eval/tasks/anli/anli_r2.yaml' 2026-05-11T15:53:29,322 adding 'lm_eval/tasks/anli/anli_r3.yaml' 2026-05-11T15:53:29,325 adding 'lm_eval/tasks/arab_culture/README.md' 2026-05-11T15:53:29,326 adding 'lm_eval/tasks/arab_culture/_arab_culture.yaml' 2026-05-11T15:53:29,327 adding 'lm_eval/tasks/arab_culture/_arab_culture_gulf.yaml' 2026-05-11T15:53:29,328 adding 'lm_eval/tasks/arab_culture/_arab_culture_levant.yaml' 2026-05-11T15:53:29,329 adding 'lm_eval/tasks/arab_culture/_arab_culture_nile_valley.yaml' 2026-05-11T15:53:29,330 adding 'lm_eval/tasks/arab_culture/_arab_culture_north_africa.yaml' 2026-05-11T15:53:29,331 adding 'lm_eval/tasks/arab_culture/_default_arab_culture_mcq_template_yaml' 2026-05-11T15:53:29,333 adding 'lm_eval/tasks/arab_culture/_generate_configs.py' 2026-05-11T15:53:29,334 adding 'lm_eval/tasks/arab_culture/arab_culture_algeria.yaml' 2026-05-11T15:53:29,335 adding 'lm_eval/tasks/arab_culture/arab_culture_egypt.yaml' 2026-05-11T15:53:29,336 adding 'lm_eval/tasks/arab_culture/arab_culture_jordan.yaml' 2026-05-11T15:53:29,337 adding 'lm_eval/tasks/arab_culture/arab_culture_ksa.yaml' 2026-05-11T15:53:29,339 adding 'lm_eval/tasks/arab_culture/arab_culture_lebanon.yaml' 2026-05-11T15:53:29,340 adding 'lm_eval/tasks/arab_culture/arab_culture_libya.yaml' 2026-05-11T15:53:29,341 adding 'lm_eval/tasks/arab_culture/arab_culture_morocco.yaml' 2026-05-11T15:53:29,342 adding 'lm_eval/tasks/arab_culture/arab_culture_palestine.yaml' 2026-05-11T15:53:29,343 adding 'lm_eval/tasks/arab_culture/arab_culture_sudan.yaml' 2026-05-11T15:53:29,345 adding 'lm_eval/tasks/arab_culture/arab_culture_syria.yaml' 2026-05-11T15:53:29,346 adding 'lm_eval/tasks/arab_culture/arab_culture_tunisia.yaml' 2026-05-11T15:53:29,347 adding 'lm_eval/tasks/arab_culture/arab_culture_uae.yaml' 2026-05-11T15:53:29,348 adding 'lm_eval/tasks/arab_culture/arab_culture_yemen.yaml' 2026-05-11T15:53:29,350 adding 'lm_eval/tasks/arab_culture/prompts.py' 2026-05-11T15:53:29,352 adding 'lm_eval/tasks/arab_culture/utils_mcq.py' 2026-05-11T15:53:29,354 adding 'lm_eval/tasks/arab_culture_completion/README.md' 2026-05-11T15:53:29,355 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion.yaml' 2026-05-11T15:53:29,356 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_gulf.yaml' 2026-05-11T15:53:29,358 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_levant.yaml' 2026-05-11T15:53:29,359 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_nile_valley.yaml' 2026-05-11T15:53:29,360 adding 'lm_eval/tasks/arab_culture_completion/_arab_culture_completion_north_africa.yaml' 2026-05-11T15:53:29,361 adding 'lm_eval/tasks/arab_culture_completion/_default_arab_culture_completion_template_yaml' 2026-05-11T15:53:29,363 adding 'lm_eval/tasks/arab_culture_completion/_generate_configs.py' 2026-05-11T15:53:29,364 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_algeria.yaml' 2026-05-11T15:53:29,365 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_egypt.yaml' 2026-05-11T15:53:29,366 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_jordan.yaml' 2026-05-11T15:53:29,367 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_ksa.yaml' 2026-05-11T15:53:29,369 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_lebanon.yaml' 2026-05-11T15:53:29,370 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_libya.yaml' 2026-05-11T15:53:29,371 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_morocco.yaml' 2026-05-11T15:53:29,372 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_palestine.yaml' 2026-05-11T15:53:29,373 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_sudan.yaml' 2026-05-11T15:53:29,374 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_syria.yaml' 2026-05-11T15:53:29,375 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_tunisia.yaml' 2026-05-11T15:53:29,376 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_uae.yaml' 2026-05-11T15:53:29,377 adding 'lm_eval/tasks/arab_culture_completion/arab_culture_completion_yemen.yaml' 2026-05-11T15:53:29,379 adding 'lm_eval/tasks/arab_culture_completion/prompts.py' 2026-05-11T15:53:29,381 adding 'lm_eval/tasks/arab_culture_completion/utils_completion.py' 2026-05-11T15:53:29,384 adding 'lm_eval/tasks/arabic_leaderboard_complete/README.md' 2026-05-11T15:53:29,385 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_complete.yaml' 2026-05-11T15:53:29,387 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa.yaml' 2026-05-11T15:53:29,388 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_mcq_exams_test_ar.yaml' 2026-05-11T15:53:29,389 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_dialects.yaml' 2026-05-11T15:53:29,390 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_meta_ar_msa.yaml' 2026-05-11T15:53:29,392 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task.yaml' 2026-05-11T15:53:29,393 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task.yaml' 2026-05-11T15:53:29,394 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task.yaml' 2026-05-11T15:53:29,395 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task.yaml' 2026-05-11T15:53:29,397 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task.yaml' 2026-05-11T15:53:29,398 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/arabic_leaderboard_alghafa_multiple_choice_sentiment_task.yaml' 2026-05-11T15:53:29,399 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_alghafa/utils.py' 2026-05-11T15:53:29,401 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_exams.yaml' 2026-05-11T15:53:29,402 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/arabic_leaderboard_arabic_exams.yaml' 2026-05-11T15:53:29,403 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_exams/utils.py' 2026-05-11T15:53:29,407 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu.yaml' 2026-05-11T15:53:29,408 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_abstract_algebra.yaml' 2026-05-11T15:53:29,409 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_anatomy.yaml' 2026-05-11T15:53:29,410 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_astronomy.yaml' 2026-05-11T15:53:29,411 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_business_ethics.yaml' 2026-05-11T15:53:29,413 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:29,414 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_biology.yaml' 2026-05-11T15:53:29,415 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_chemistry.yaml' 2026-05-11T15:53:29,416 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_computer_science.yaml' 2026-05-11T15:53:29,417 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_mathematics.yaml' 2026-05-11T15:53:29,419 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_medicine.yaml' 2026-05-11T15:53:29,420 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_college_physics.yaml' 2026-05-11T15:53:29,421 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_computer_security.yaml' 2026-05-11T15:53:29,423 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_conceptual_physics.yaml' 2026-05-11T15:53:29,424 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_econometrics.yaml' 2026-05-11T15:53:29,425 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_electrical_engineering.yaml' 2026-05-11T15:53:29,426 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:29,427 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_formal_logic.yaml' 2026-05-11T15:53:29,429 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_global_facts.yaml' 2026-05-11T15:53:29,430 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_biology.yaml' 2026-05-11T15:53:29,431 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:29,432 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:29,433 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_european_history.yaml' 2026-05-11T15:53:29,435 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_geography.yaml' 2026-05-11T15:53:29,436 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:29,437 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:29,438 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:29,440 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:29,441 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_physics.yaml' 2026-05-11T15:53:29,442 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_psychology.yaml' 2026-05-11T15:53:29,443 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_statistics.yaml' 2026-05-11T15:53:29,444 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_us_history.yaml' 2026-05-11T15:53:29,445 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_high_school_world_history.yaml' 2026-05-11T15:53:29,447 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_aging.yaml' 2026-05-11T15:53:29,448 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_human_sexuality.yaml' 2026-05-11T15:53:29,449 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_international_law.yaml' 2026-05-11T15:53:29,450 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_jurisprudence.yaml' 2026-05-11T15:53:29,452 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_logical_fallacies.yaml' 2026-05-11T15:53:29,453 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_machine_learning.yaml' 2026-05-11T15:53:29,454 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_management.yaml' 2026-05-11T15:53:29,456 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_marketing.yaml' 2026-05-11T15:53:29,457 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_medical_genetics.yaml' 2026-05-11T15:53:29,458 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_miscellaneous.yaml' 2026-05-11T15:53:29,459 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_disputes.yaml' 2026-05-11T15:53:29,461 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_moral_scenarios.yaml' 2026-05-11T15:53:29,462 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_nutrition.yaml' 2026-05-11T15:53:29,463 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_philosophy.yaml' 2026-05-11T15:53:29,465 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_prehistory.yaml' 2026-05-11T15:53:29,466 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_accounting.yaml' 2026-05-11T15:53:29,467 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_law.yaml' 2026-05-11T15:53:29,468 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_medicine.yaml' 2026-05-11T15:53:29,470 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_professional_psychology.yaml' 2026-05-11T15:53:29,471 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_public_relations.yaml' 2026-05-11T15:53:29,472 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_security_studies.yaml' 2026-05-11T15:53:29,473 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_sociology.yaml' 2026-05-11T15:53:29,474 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:29,476 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_virology.yaml' 2026-05-11T15:53:29,477 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/arabic_leaderboard_arabic_mmlu_world_religions.yaml' 2026-05-11T15:53:29,478 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mmlu/utils.py' 2026-05-11T15:53:29,480 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_leaderboard_arabic_mt_arc_challenge.yaml' 2026-05-11T15:53:29,481 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/arabic_mt_arc_challenge.yaml' 2026-05-11T15:53:29,482 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_challenge/utils.py' 2026-05-11T15:53:29,484 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_leaderboard_arabic_mt_arc_easy.yaml' 2026-05-11T15:53:29,485 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/arabic_mt_arc_easy.yaml' 2026-05-11T15:53:29,487 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_arc_easy/utils.py' 2026-05-11T15:53:29,488 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_leaderboard_arabic_mt_boolq.yaml' 2026-05-11T15:53:29,490 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/arabic_mt_boolq.yaml' 2026-05-11T15:53:29,491 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_boolq/utils.py' 2026-05-11T15:53:29,493 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_leaderboard_arabic_mt_copa.yaml' 2026-05-11T15:53:29,494 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/arabic_mt_copa.yaml' 2026-05-11T15:53:29,495 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_copa/utils.py' 2026-05-11T15:53:29,497 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_leaderboard_arabic_mt_hellaswag.yaml' 2026-05-11T15:53:29,498 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/arabic_mt_hellaswag.yaml' 2026-05-11T15:53:29,500 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_hellaswag/utils.py' 2026-05-11T15:53:29,501 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_leaderboard_arabic_mt_mmlu.yaml' 2026-05-11T15:53:29,503 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/arabic_mt_mmlu.yaml' 2026-05-11T15:53:29,504 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_mmlu/utils.py' 2026-05-11T15:53:29,506 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_leaderboard_arabic_mt_openbook_qa.yaml' 2026-05-11T15:53:29,507 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/arabic_mt_openbook_qa.yaml' 2026-05-11T15:53:29,508 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_openbook_qa/utils.py' 2026-05-11T15:53:29,510 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_leaderboard_arabic_mt_piqa.yaml' 2026-05-11T15:53:29,511 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/arabic_mt_piqa.yaml' 2026-05-11T15:53:29,512 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_piqa/utils.py' 2026-05-11T15:53:29,514 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_leaderboard_arabic_mt_race.yaml' 2026-05-11T15:53:29,515 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/arabic_mt_race.yaml' 2026-05-11T15:53:29,516 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_race/utils.py' 2026-05-11T15:53:29,518 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_leaderboard_arabic_mt_sciq.yaml' 2026-05-11T15:53:29,519 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/arabic_mt_sciq.yaml' 2026-05-11T15:53:29,521 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_sciq/utils.py' 2026-05-11T15:53:29,522 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_leaderboard_arabic_mt_toxigen.yaml' 2026-05-11T15:53:29,524 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/arabic_mt_toxigen.yaml' 2026-05-11T15:53:29,525 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_arabic_mt_toxigen/utils.py' 2026-05-11T15:53:29,528 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva.yaml' 2026-05-11T15:53:29,529 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Algeria.yaml' 2026-05-11T15:53:29,530 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Ancient_Egypt.yaml' 2026-05-11T15:53:29,531 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arab_Empire.yaml' 2026-05-11T15:53:29,533 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Architecture.yaml' 2026-05-11T15:53:29,534 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Art.yaml' 2026-05-11T15:53:29,535 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Astronomy.yaml' 2026-05-11T15:53:29,536 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Calligraphy.yaml' 2026-05-11T15:53:29,538 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ceremony.yaml' 2026-05-11T15:53:29,539 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Clothing.yaml' 2026-05-11T15:53:29,540 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Culture.yaml' 2026-05-11T15:53:29,541 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Food.yaml' 2026-05-11T15:53:29,542 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Funeral.yaml' 2026-05-11T15:53:29,544 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Geography.yaml' 2026-05-11T15:53:29,545 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_History.yaml' 2026-05-11T15:53:29,546 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Language_Origin.yaml' 2026-05-11T15:53:29,547 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Literature.yaml' 2026-05-11T15:53:29,549 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Math.yaml' 2026-05-11T15:53:29,550 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Medicine.yaml' 2026-05-11T15:53:29,551 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Music.yaml' 2026-05-11T15:53:29,552 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Ornament.yaml' 2026-05-11T15:53:29,554 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Philosophy.yaml' 2026-05-11T15:53:29,555 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry.yaml' 2026-05-11T15:53:29,556 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Arabic_Wedding.yaml' 2026-05-11T15:53:29,557 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Bahrain.yaml' 2026-05-11T15:53:29,559 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Comoros.yaml' 2026-05-11T15:53:29,560 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Egypt_modern.yaml' 2026-05-11T15:53:29,561 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromAncientEgypt.yaml' 2026-05-11T15:53:29,562 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromByzantium.yaml' 2026-05-11T15:53:29,564 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromChina.yaml' 2026-05-11T15:53:29,565 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromGreece.yaml' 2026-05-11T15:53:29,567 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromIslam.yaml' 2026-05-11T15:53:29,568 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromPersia.yaml' 2026-05-11T15:53:29,569 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_InfluenceFromRome.yaml' 2026-05-11T15:53:29,571 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Iraq.yaml' 2026-05-11T15:53:29,572 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_Education.yaml' 2026-05-11T15:53:29,574 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islam_branches_and_schools.yaml' 2026-05-11T15:53:29,575 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Islamic_law_system.yaml' 2026-05-11T15:53:29,576 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Jordan.yaml' 2026-05-11T15:53:29,577 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Kuwait.yaml' 2026-05-11T15:53:29,579 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Lebanon.yaml' 2026-05-11T15:53:29,580 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Libya.yaml' 2026-05-11T15:53:29,581 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mauritania.yaml' 2026-05-11T15:53:29,582 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Mesopotamia_civilization.yaml' 2026-05-11T15:53:29,584 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Morocco.yaml' 2026-05-11T15:53:29,585 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Oman.yaml' 2026-05-11T15:53:29,587 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Palestine.yaml' 2026-05-11T15:53:29,588 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Qatar.yaml' 2026-05-11T15:53:29,589 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Saudi_Arabia.yaml' 2026-05-11T15:53:29,590 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Somalia.yaml' 2026-05-11T15:53:29,592 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Sudan.yaml' 2026-05-11T15:53:29,593 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Syria.yaml' 2026-05-11T15:53:29,594 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Tunisia.yaml' 2026-05-11T15:53:29,595 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_United_Arab_Emirates.yaml' 2026-05-11T15:53:29,596 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_Yemen.yaml' 2026-05-11T15:53:29,598 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_communication.yaml' 2026-05-11T15:53:29,599 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_computer_and_phone.yaml' 2026-05-11T15:53:29,600 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_daily_life.yaml' 2026-05-11T15:53:29,601 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/arabic_leaderboard_acva_entertainment.yaml' 2026-05-11T15:53:29,603 adding 'lm_eval/tasks/arabic_leaderboard_complete/arabic_leaderboard_avca/utils.py' 2026-05-11T15:53:29,604 adding 'lm_eval/tasks/arabic_leaderboard_light/README.md' 2026-05-11T15:53:29,606 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_light.yaml' 2026-05-11T15:53:29,608 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_light.yaml' 2026-05-11T15:53:29,609 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_mcq_exams_test_ar_light.yaml' 2026-05-11T15:53:29,610 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_dialects_light.yaml' 2026-05-11T15:53:29,611 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_meta_ar_msa_light.yaml' 2026-05-11T15:53:29,613 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_facts_truefalse_balanced_task_light.yaml' 2026-05-11T15:53:29,614 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_soqal_task_light.yaml' 2026-05-11T15:53:29,615 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_grounded_statement_xglue_mlqa_task_light.yaml' 2026-05-11T15:53:29,616 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_no_neutral_task_light.yaml' 2026-05-11T15:53:29,618 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_rating_sentiment_task_light.yaml' 2026-05-11T15:53:29,619 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/arabic_leaderboard_alghafa_multiple_choice_sentiment_task_light.yaml' 2026-05-11T15:53:29,620 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_alghafa_light/utils.py' 2026-05-11T15:53:29,622 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_exams_light.yaml' 2026-05-11T15:53:29,623 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/arabic_leaderboard_arabic_exams_light.yaml' 2026-05-11T15:53:29,624 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_exams_light/utils.py' 2026-05-11T15:53:29,628 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_abstract_algebra_light.yaml' 2026-05-11T15:53:29,629 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_anatomy_light.yaml' 2026-05-11T15:53:29,630 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_astronomy_light.yaml' 2026-05-11T15:53:29,631 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_business_ethics_light.yaml' 2026-05-11T15:53:29,632 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_clinical_knowledge_light.yaml' 2026-05-11T15:53:29,634 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_biology_light.yaml' 2026-05-11T15:53:29,635 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_chemistry_light.yaml' 2026-05-11T15:53:29,636 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_computer_science_light.yaml' 2026-05-11T15:53:29,637 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_mathematics_light.yaml' 2026-05-11T15:53:29,639 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_medicine_light.yaml' 2026-05-11T15:53:29,640 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_college_physics_light.yaml' 2026-05-11T15:53:29,641 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_computer_security_light.yaml' 2026-05-11T15:53:29,642 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_conceptual_physics_light.yaml' 2026-05-11T15:53:29,643 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_econometrics_light.yaml' 2026-05-11T15:53:29,645 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_electrical_engineering_light.yaml' 2026-05-11T15:53:29,646 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_elementary_mathematics_light.yaml' 2026-05-11T15:53:29,647 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_formal_logic_light.yaml' 2026-05-11T15:53:29,648 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_global_facts_light.yaml' 2026-05-11T15:53:29,650 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_biology_light.yaml' 2026-05-11T15:53:29,651 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_chemistry_light.yaml' 2026-05-11T15:53:29,653 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_computer_science_light.yaml' 2026-05-11T15:53:29,654 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_european_history_light.yaml' 2026-05-11T15:53:29,655 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_geography_light.yaml' 2026-05-11T15:53:29,656 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_government_and_politics_light.yaml' 2026-05-11T15:53:29,658 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_macroeconomics_light.yaml' 2026-05-11T15:53:29,659 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_mathematics_light.yaml' 2026-05-11T15:53:29,660 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_microeconomics_light.yaml' 2026-05-11T15:53:29,661 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_physics_light.yaml' 2026-05-11T15:53:29,662 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_psychology_light.yaml' 2026-05-11T15:53:29,664 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_statistics_light.yaml' 2026-05-11T15:53:29,665 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_us_history_light.yaml' 2026-05-11T15:53:29,666 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_high_school_world_history_light.yaml' 2026-05-11T15:53:29,667 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_aging_light.yaml' 2026-05-11T15:53:29,668 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_human_sexuality_light.yaml' 2026-05-11T15:53:29,670 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_international_law_light.yaml' 2026-05-11T15:53:29,671 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_jurisprudence_light.yaml' 2026-05-11T15:53:29,672 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_light.yaml' 2026-05-11T15:53:29,673 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_logical_fallacies_light.yaml' 2026-05-11T15:53:29,675 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_machine_learning_light.yaml' 2026-05-11T15:53:29,676 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_management_light.yaml' 2026-05-11T15:53:29,677 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_marketing_light.yaml' 2026-05-11T15:53:29,678 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_medical_genetics_light.yaml' 2026-05-11T15:53:29,680 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_miscellaneous_light.yaml' 2026-05-11T15:53:29,681 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_disputes_light.yaml' 2026-05-11T15:53:29,682 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_moral_scenarios_light.yaml' 2026-05-11T15:53:29,684 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_nutrition_light.yaml' 2026-05-11T15:53:29,685 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_philosophy_light.yaml' 2026-05-11T15:53:29,686 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_prehistory_light.yaml' 2026-05-11T15:53:29,688 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_accounting_light.yaml' 2026-05-11T15:53:29,689 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_law_light.yaml' 2026-05-11T15:53:29,690 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_medicine_light.yaml' 2026-05-11T15:53:29,691 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_professional_psychology_light.yaml' 2026-05-11T15:53:29,692 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_public_relations_light.yaml' 2026-05-11T15:53:29,694 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_security_studies_light.yaml' 2026-05-11T15:53:29,695 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_sociology_light.yaml' 2026-05-11T15:53:29,696 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_us_foreign_policy_light.yaml' 2026-05-11T15:53:29,697 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_virology_light.yaml' 2026-05-11T15:53:29,699 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/arabic_leaderboard_arabic_mmlu_world_religions_light.yaml' 2026-05-11T15:53:29,700 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mmlu_light/utils.py' 2026-05-11T15:53:29,702 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_leaderboard_arabic_mt_arc_challenge_light.yaml' 2026-05-11T15:53:29,703 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/arabic_mt_arc_challenge_light.yaml' 2026-05-11T15:53:29,704 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_challenge_light/utils.py' 2026-05-11T15:53:29,706 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_leaderboard_arabic_mt_arc_easy_light.yaml' 2026-05-11T15:53:29,707 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/arabic_mt_arc_easy_light.yaml' 2026-05-11T15:53:29,709 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_arc_easy_light/utils.py' 2026-05-11T15:53:29,711 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_leaderboard_arabic_mt_boolq_light.yaml' 2026-05-11T15:53:29,712 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/arabic_mt_boolq_light.yaml' 2026-05-11T15:53:29,713 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_boolq_light/utils.py' 2026-05-11T15:53:29,715 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arabic_mt_copa_light.yaml' 2026-05-11T15:53:29,716 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/arbic_leaderboard_arabic_mt_copa_light.yaml' 2026-05-11T15:53:29,718 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_copa_light/utils.py' 2026-05-11T15:53:29,719 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_leaderboard_arabic_mt_hellaswag_light.yaml' 2026-05-11T15:53:29,721 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/arabic_mt_hellaswag_light.yaml' 2026-05-11T15:53:29,722 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_hellaswag_light/utils.py' 2026-05-11T15:53:29,724 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_leaderboard_arabic_mt_mmlu_light.yaml' 2026-05-11T15:53:29,725 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/arabic_mt_mmlu_light.yaml' 2026-05-11T15:53:29,726 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_mmlu_light/utils.py' 2026-05-11T15:53:29,728 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_leaderboard_arabic_mt_openbook_qa_light.yaml' 2026-05-11T15:53:29,730 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/arabic_mt_openbook_qa_light.yaml' 2026-05-11T15:53:29,731 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_openbook_qa_light/utils.py' 2026-05-11T15:53:29,733 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_leaderboard_arabic_mt_piqa_light.yaml' 2026-05-11T15:53:29,734 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/arabic_mt_piqa_light.yaml' 2026-05-11T15:53:29,735 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_piqa_light/utils.py' 2026-05-11T15:53:29,737 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_leaderboard_arabic_mt_race_light.yaml' 2026-05-11T15:53:29,739 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/arabic_mt_race_light.yaml' 2026-05-11T15:53:29,740 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_race_light/utils.py' 2026-05-11T15:53:29,742 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_leaderboard_arabic_mt_sciq_light.yaml' 2026-05-11T15:53:29,743 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/arabic_mt_sciq_light.yaml' 2026-05-11T15:53:29,744 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_sciq_light/utils.py' 2026-05-11T15:53:29,746 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_leaderboard_arabic_mt_toxigen_light.yaml' 2026-05-11T15:53:29,748 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/arabic_mt_toxigen_light.yaml' 2026-05-11T15:53:29,749 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_arabic_mt_toxigen_light/utils.py' 2026-05-11T15:53:29,752 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Algeria_light.yaml' 2026-05-11T15:53:29,753 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Ancient_Egypt_light.yaml' 2026-05-11T15:53:29,755 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arab_Empire_light.yaml' 2026-05-11T15:53:29,756 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Architecture_light.yaml' 2026-05-11T15:53:29,758 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Art_light.yaml' 2026-05-11T15:53:29,759 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Astronomy_light.yaml' 2026-05-11T15:53:29,760 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Calligraphy_light.yaml' 2026-05-11T15:53:29,762 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ceremony_light.yaml' 2026-05-11T15:53:29,763 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Clothing_light.yaml' 2026-05-11T15:53:29,764 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Culture_light.yaml' 2026-05-11T15:53:29,765 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Food_light.yaml' 2026-05-11T15:53:29,767 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Funeral_light.yaml' 2026-05-11T15:53:29,768 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Geography_light.yaml' 2026-05-11T15:53:29,769 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_History_light.yaml' 2026-05-11T15:53:29,771 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Language_Origin_light.yaml' 2026-05-11T15:53:29,772 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Literature_light.yaml' 2026-05-11T15:53:29,773 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Math_light.yaml' 2026-05-11T15:53:29,774 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Medicine_light.yaml' 2026-05-11T15:53:29,776 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Music_light.yaml' 2026-05-11T15:53:29,777 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Ornament_light.yaml' 2026-05-11T15:53:29,778 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Philosophy_light.yaml' 2026-05-11T15:53:29,780 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Physics_and_Chemistry_light.yaml' 2026-05-11T15:53:29,781 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Arabic_Wedding_light.yaml' 2026-05-11T15:53:29,783 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Bahrain_light.yaml' 2026-05-11T15:53:29,784 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Comoros_light.yaml' 2026-05-11T15:53:29,786 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Egypt_modern_light.yaml' 2026-05-11T15:53:29,787 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromAncientEgypt_light.yaml' 2026-05-11T15:53:29,788 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromByzantium_light.yaml' 2026-05-11T15:53:29,790 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromChina_light.yaml' 2026-05-11T15:53:29,791 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromGreece_light.yaml' 2026-05-11T15:53:29,792 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromIslam_light.yaml' 2026-05-11T15:53:29,794 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromPersia_light.yaml' 2026-05-11T15:53:29,795 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_InfluenceFromRome_light.yaml' 2026-05-11T15:53:29,796 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Iraq_light.yaml' 2026-05-11T15:53:29,798 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_Education_light.yaml' 2026-05-11T15:53:29,799 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islam_branches_and_schools_light.yaml' 2026-05-11T15:53:29,800 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Islamic_law_system_light.yaml' 2026-05-11T15:53:29,801 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Jordan_light.yaml' 2026-05-11T15:53:29,803 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Kuwait_light.yaml' 2026-05-11T15:53:29,804 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Lebanon_light.yaml' 2026-05-11T15:53:29,805 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Libya_light.yaml' 2026-05-11T15:53:29,806 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mauritania_light.yaml' 2026-05-11T15:53:29,807 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Mesopotamia_civilization_light.yaml' 2026-05-11T15:53:29,809 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Morocco_light.yaml' 2026-05-11T15:53:29,810 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Oman_light.yaml' 2026-05-11T15:53:29,811 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Palestine_light.yaml' 2026-05-11T15:53:29,813 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Qatar_light.yaml' 2026-05-11T15:53:29,814 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Saudi_Arabia_light.yaml' 2026-05-11T15:53:29,815 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Somalia_light.yaml' 2026-05-11T15:53:29,816 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Sudan_light.yaml' 2026-05-11T15:53:29,818 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Syria_light.yaml' 2026-05-11T15:53:29,819 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Tunisia_light.yaml' 2026-05-11T15:53:29,820 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_United_Arab_Emirates_light.yaml' 2026-05-11T15:53:29,821 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_Yemen_light.yaml' 2026-05-11T15:53:29,823 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_communication_light.yaml' 2026-05-11T15:53:29,824 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_computer_and_phone_light.yaml' 2026-05-11T15:53:29,825 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_daily_life_light.yaml' 2026-05-11T15:53:29,826 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_entertainment_light.yaml' 2026-05-11T15:53:29,828 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/arabic_leaderboard_acva_light.yaml' 2026-05-11T15:53:29,829 adding 'lm_eval/tasks/arabic_leaderboard_light/arabic_leaderboard_avca_light/utils.py' 2026-05-11T15:53:29,832 adding 'lm_eval/tasks/arabicmmlu/README.md' 2026-05-11T15:53:29,834 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu.yaml' 2026-05-11T15:53:29,835 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_humanities.yaml' 2026-05-11T15:53:29,836 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_language.yaml' 2026-05-11T15:53:29,837 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_other.yaml' 2026-05-11T15:53:29,839 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_social_science.yaml' 2026-05-11T15:53:29,840 adding 'lm_eval/tasks/arabicmmlu/_arabicmmlu_stem.yaml' 2026-05-11T15:53:29,841 adding 'lm_eval/tasks/arabicmmlu/_default_arabicmmlu_template_yaml' 2026-05-11T15:53:29,843 adding 'lm_eval/tasks/arabicmmlu/_generate_configs.py' 2026-05-11T15:53:29,844 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_accounting_university.yaml' 2026-05-11T15:53:29,845 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_general.yaml' 2026-05-11T15:53:29,846 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_grammar.yaml' 2026-05-11T15:53:29,847 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_high_school.yaml' 2026-05-11T15:53:29,848 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_middle_school.yaml' 2026-05-11T15:53:29,849 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_arabic_language_primary_school.yaml' 2026-05-11T15:53:29,851 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_biology_high_school.yaml' 2026-05-11T15:53:29,852 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_civics_high_school.yaml' 2026-05-11T15:53:29,853 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_civics_middle_school.yaml' 2026-05-11T15:53:29,854 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_high_school.yaml' 2026-05-11T15:53:29,856 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_middle_school.yaml' 2026-05-11T15:53:29,857 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_primary_school.yaml' 2026-05-11T15:53:29,858 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_computer_science_university.yaml' 2026-05-11T15:53:29,859 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_driving_test.yaml' 2026-05-11T15:53:29,860 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_economics_high_school.yaml' 2026-05-11T15:53:29,862 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_economics_middle_school.yaml' 2026-05-11T15:53:29,863 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_economics_university.yaml' 2026-05-11T15:53:29,864 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge.yaml' 2026-05-11T15:53:29,865 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_middle_school.yaml' 2026-05-11T15:53:29,866 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_general_knowledge_primary_school.yaml' 2026-05-11T15:53:29,868 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_geography_high_school.yaml' 2026-05-11T15:53:29,869 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_geography_middle_school.yaml' 2026-05-11T15:53:29,871 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_geography_primary_school.yaml' 2026-05-11T15:53:29,872 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_history_high_school.yaml' 2026-05-11T15:53:29,873 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_history_middle_school.yaml' 2026-05-11T15:53:29,874 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_history_primary_school.yaml' 2026-05-11T15:53:29,875 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies.yaml' 2026-05-11T15:53:29,877 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_high_school.yaml' 2026-05-11T15:53:29,878 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_middle_school.yaml' 2026-05-11T15:53:29,879 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_islamic_studies_primary_school.yaml' 2026-05-11T15:53:29,880 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_law_professional.yaml' 2026-05-11T15:53:29,881 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_management_university.yaml' 2026-05-11T15:53:29,883 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_math_primary_school.yaml' 2026-05-11T15:53:29,884 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_middle_school.yaml' 2026-05-11T15:53:29,885 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_natural_science_primary_school.yaml' 2026-05-11T15:53:29,886 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_philosophy_high_school.yaml' 2026-05-11T15:53:29,888 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_physics_high_school.yaml' 2026-05-11T15:53:29,889 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_political_science_university.yaml' 2026-05-11T15:53:29,890 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_middle_school.yaml' 2026-05-11T15:53:29,891 adding 'lm_eval/tasks/arabicmmlu/arabicmmlu_social_science_primary_school.yaml' 2026-05-11T15:53:29,892 adding 'lm_eval/tasks/arabicmmlu/utils.py' 2026-05-11T15:53:29,894 adding 'lm_eval/tasks/aradice/README.md' 2026-05-11T15:53:29,896 adding 'lm_eval/tasks/aradice/aradice.yaml' 2026-05-11T15:53:29,899 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU.yaml' 2026-05-11T15:53:29,900 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_history.yaml' 2026-05-11T15:53:29,901 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,902 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml' 2026-05-11T15:53:29,903 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml' 2026-05-11T15:53:29,905 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_civics.yaml' 2026-05-11T15:53:29,906 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_economics.yaml' 2026-05-11T15:53:29,907 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_social-science_geography.yaml' 2026-05-11T15:53:29,908 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_biology.yaml' 2026-05-11T15:53:29,910 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml' 2026-05-11T15:53:29,911 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_high_stem_physics.yaml' 2026-05-11T15:53:29,912 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_history.yaml' 2026-05-11T15:53:29,913 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,914 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml' 2026-05-11T15:53:29,916 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml' 2026-05-11T15:53:29,917 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml' 2026-05-11T15:53:29,918 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml' 2026-05-11T15:53:29,919 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml' 2026-05-11T15:53:29,921 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml' 2026-05-11T15:53:29,922 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml' 2026-05-11T15:53:29,923 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml' 2026-05-11T15:53:29,924 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,925 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml' 2026-05-11T15:53:29,927 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml' 2026-05-11T15:53:29,928 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_driving-test.yaml' 2026-05-11T15:53:29,929 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml' 2026-05-11T15:53:29,930 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_history.yaml' 2026-05-11T15:53:29,932 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,933 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml' 2026-05-11T15:53:29,934 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml' 2026-05-11T15:53:29,935 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml' 2026-05-11T15:53:29,936 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml' 2026-05-11T15:53:29,937 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml' 2026-05-11T15:53:29,939 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_math.yaml' 2026-05-11T15:53:29,940 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml' 2026-05-11T15:53:29,941 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_prof_humanities_law.yaml' 2026-05-11T15:53:29,943 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_other_management.yaml' 2026-05-11T15:53:29,944 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml' 2026-05-11T15:53:29,945 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml' 2026-05-11T15:53:29,946 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml' 2026-05-11T15:53:29,947 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml' 2026-05-11T15:53:29,949 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/_default_template_yaml' 2026-05-11T15:53:29,950 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/metrics.py' 2026-05-11T15:53:29,951 adding 'lm_eval/tasks/aradice/ArabicMMLU/EGY/utils.py' 2026-05-11T15:53:29,954 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU.yaml' 2026-05-11T15:53:29,955 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_history.yaml' 2026-05-11T15:53:29,957 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,958 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_humanities_philosophy.yaml' 2026-05-11T15:53:29,959 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_language_arabic-language.yaml' 2026-05-11T15:53:29,960 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_civics.yaml' 2026-05-11T15:53:29,961 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_economics.yaml' 2026-05-11T15:53:29,963 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_social-science_geography.yaml' 2026-05-11T15:53:29,964 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_biology.yaml' 2026-05-11T15:53:29,965 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_computer-science.yaml' 2026-05-11T15:53:29,966 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_high_stem_physics.yaml' 2026-05-11T15:53:29,968 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_history.yaml' 2026-05-11T15:53:29,969 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,970 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_language_arabic-language.yaml' 2026-05-11T15:53:29,971 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_other_general-knowledge.yaml' 2026-05-11T15:53:29,973 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_civics.yaml' 2026-05-11T15:53:29,974 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_economics.yaml' 2026-05-11T15:53:29,975 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_geography.yaml' 2026-05-11T15:53:29,977 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_social-science_social-science.yaml' 2026-05-11T15:53:29,978 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_computer-science.yaml' 2026-05-11T15:53:29,979 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_middle_stem_natural-science.yaml' 2026-05-11T15:53:29,980 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,981 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-general.yaml' 2026-05-11T15:53:29,983 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_language_arabic-language-grammar.yaml' 2026-05-11T15:53:29,984 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_driving-test.yaml' 2026-05-11T15:53:29,985 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_na_other_general-knowledge.yaml' 2026-05-11T15:53:29,987 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_history.yaml' 2026-05-11T15:53:29,988 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_humanities_islamic-studies.yaml' 2026-05-11T15:53:29,989 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_language_arabic-language.yaml' 2026-05-11T15:53:29,990 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_other_general-knowledge.yaml' 2026-05-11T15:53:29,991 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_geography.yaml' 2026-05-11T15:53:29,993 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_social-science_social-science.yaml' 2026-05-11T15:53:29,994 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_computer-science.yaml' 2026-05-11T15:53:29,995 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_math.yaml' 2026-05-11T15:53:29,996 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_primary_stem_natural-science.yaml' 2026-05-11T15:53:29,997 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_prof_humanities_law.yaml' 2026-05-11T15:53:29,999 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_other_management.yaml' 2026-05-11T15:53:30,000 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_accounting.yaml' 2026-05-11T15:53:30,001 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_economics.yaml' 2026-05-11T15:53:30,002 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_social-science_political-science.yaml' 2026-05-11T15:53:30,004 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/AraDiCE_ArabicMMLU_univ_stem_computer-science.yaml' 2026-05-11T15:53:30,005 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/_default_template_yaml' 2026-05-11T15:53:30,006 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/metrics.py' 2026-05-11T15:53:30,008 adding 'lm_eval/tasks/aradice/ArabicMMLU/LEV/utils.py' 2026-05-11T15:53:30,010 adding 'lm_eval/tasks/aradice/boolq/EGY/boolq_egy.yaml' 2026-05-11T15:53:30,012 adding 'lm_eval/tasks/aradice/boolq/EGY/metrics.py' 2026-05-11T15:53:30,013 adding 'lm_eval/tasks/aradice/boolq/EGY/utils.py' 2026-05-11T15:53:30,015 adding 'lm_eval/tasks/aradice/boolq/ENG/boolq_eng.yaml' 2026-05-11T15:53:30,016 adding 'lm_eval/tasks/aradice/boolq/ENG/metrics.py' 2026-05-11T15:53:30,017 adding 'lm_eval/tasks/aradice/boolq/ENG/utils.py' 2026-05-11T15:53:30,019 adding 'lm_eval/tasks/aradice/boolq/LEV/boolq_lev.yaml' 2026-05-11T15:53:30,020 adding 'lm_eval/tasks/aradice/boolq/LEV/metrics.py' 2026-05-11T15:53:30,022 adding 'lm_eval/tasks/aradice/boolq/LEV/utils.py' 2026-05-11T15:53:30,024 adding 'lm_eval/tasks/aradice/boolq/MSA/boolq_msa.yaml' 2026-05-11T15:53:30,025 adding 'lm_eval/tasks/aradice/boolq/MSA/metrics.py' 2026-05-11T15:53:30,026 adding 'lm_eval/tasks/aradice/boolq/MSA/utils.py' 2026-05-11T15:53:30,028 adding 'lm_eval/tasks/aradice/cultural-benchmark/egypt.yaml' 2026-05-11T15:53:30,029 adding 'lm_eval/tasks/aradice/cultural-benchmark/jordan.yaml' 2026-05-11T15:53:30,031 adding 'lm_eval/tasks/aradice/cultural-benchmark/lebanon.yaml' 2026-05-11T15:53:30,032 adding 'lm_eval/tasks/aradice/cultural-benchmark/metrics.py' 2026-05-11T15:53:30,033 adding 'lm_eval/tasks/aradice/cultural-benchmark/palestine.yaml' 2026-05-11T15:53:30,034 adding 'lm_eval/tasks/aradice/cultural-benchmark/qatar.yaml' 2026-05-11T15:53:30,035 adding 'lm_eval/tasks/aradice/cultural-benchmark/syria.yaml' 2026-05-11T15:53:30,037 adding 'lm_eval/tasks/aradice/cultural-benchmark/utils.py' 2026-05-11T15:53:30,039 adding 'lm_eval/tasks/aradice/openbookqa/metrics.py' 2026-05-11T15:53:30,040 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_egy.yaml' 2026-05-11T15:53:30,041 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_eng.yaml' 2026-05-11T15:53:30,042 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_lev.yaml' 2026-05-11T15:53:30,043 adding 'lm_eval/tasks/aradice/openbookqa/openbookqa_msa.yaml' 2026-05-11T15:53:30,045 adding 'lm_eval/tasks/aradice/openbookqa/utils.py' 2026-05-11T15:53:30,047 adding 'lm_eval/tasks/aradice/piqa/metrics.py' 2026-05-11T15:53:30,048 adding 'lm_eval/tasks/aradice/piqa/piqa_egy.yaml' 2026-05-11T15:53:30,049 adding 'lm_eval/tasks/aradice/piqa/piqa_eng.yaml' 2026-05-11T15:53:30,050 adding 'lm_eval/tasks/aradice/piqa/piqa_lev.yaml' 2026-05-11T15:53:30,051 adding 'lm_eval/tasks/aradice/piqa/piqa_msa.yaml' 2026-05-11T15:53:30,053 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/metrics.py' 2026-05-11T15:53:30,054 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_egy.yaml' 2026-05-11T15:53:30,056 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_eng.yaml' 2026-05-11T15:53:30,057 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_lev.yaml' 2026-05-11T15:53:30,058 adding 'lm_eval/tasks/aradice/truthfulqa_mcq/truthfulqa_mc1_msa.yaml' 2026-05-11T15:53:30,060 adding 'lm_eval/tasks/aradice/winogrande/metrics.py' 2026-05-11T15:53:30,061 adding 'lm_eval/tasks/aradice/winogrande/utils.py' 2026-05-11T15:53:30,063 adding 'lm_eval/tasks/aradice/winogrande/winogrande_egy.yaml' 2026-05-11T15:53:30,064 adding 'lm_eval/tasks/aradice/winogrande/winogrande_eng.yaml' 2026-05-11T15:53:30,065 adding 'lm_eval/tasks/aradice/winogrande/winogrande_lev.yaml' 2026-05-11T15:53:30,066 adding 'lm_eval/tasks/aradice/winogrande/winogrande_msa.yaml' 2026-05-11T15:53:30,068 adding 'lm_eval/tasks/arc/README.md' 2026-05-11T15:53:30,070 adding 'lm_eval/tasks/arc/arc_challenge.yaml' 2026-05-11T15:53:30,071 adding 'lm_eval/tasks/arc/arc_challenge_chat.yaml' 2026-05-11T15:53:30,072 adding 'lm_eval/tasks/arc/arc_easy.yaml' 2026-05-11T15:53:30,074 adding 'lm_eval/tasks/arc/groups.yaml' 2026-05-11T15:53:30,075 adding 'lm_eval/tasks/arc/test_arc.yaml' 2026-05-11T15:53:30,077 adding 'lm_eval/tasks/arc_mt/README.md' 2026-05-11T15:53:30,078 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_da.yaml' 2026-05-11T15:53:30,079 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_de.yaml' 2026-05-11T15:53:30,080 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_el.yaml' 2026-05-11T15:53:30,081 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_es.yaml' 2026-05-11T15:53:30,083 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_fi.yaml' 2026-05-11T15:53:30,084 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_hu.yaml' 2026-05-11T15:53:30,085 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_is.yaml' 2026-05-11T15:53:30,086 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_it.yaml' 2026-05-11T15:53:30,088 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_nb.yaml' 2026-05-11T15:53:30,089 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_pl.yaml' 2026-05-11T15:53:30,090 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_pt.yaml' 2026-05-11T15:53:30,091 adding 'lm_eval/tasks/arc_mt/arc_challenge_mt_sv.yaml' 2026-05-11T15:53:30,093 adding 'lm_eval/tasks/arithmetic/README.md' 2026-05-11T15:53:30,094 adding 'lm_eval/tasks/arithmetic/arithmetic_1dc.yaml' 2026-05-11T15:53:30,096 adding 'lm_eval/tasks/arithmetic/arithmetic_2da.yaml' 2026-05-11T15:53:30,097 adding 'lm_eval/tasks/arithmetic/arithmetic_2dm.yaml' 2026-05-11T15:53:30,098 adding 'lm_eval/tasks/arithmetic/arithmetic_2ds.yaml' 2026-05-11T15:53:30,099 adding 'lm_eval/tasks/arithmetic/arithmetic_3da.yaml' 2026-05-11T15:53:30,100 adding 'lm_eval/tasks/arithmetic/arithmetic_3ds.yaml' 2026-05-11T15:53:30,101 adding 'lm_eval/tasks/arithmetic/arithmetic_4da.yaml' 2026-05-11T15:53:30,102 adding 'lm_eval/tasks/arithmetic/arithmetic_4ds.yaml' 2026-05-11T15:53:30,103 adding 'lm_eval/tasks/arithmetic/arithmetic_5da.yaml' 2026-05-11T15:53:30,104 adding 'lm_eval/tasks/arithmetic/arithmetic_5ds.yaml' 2026-05-11T15:53:30,106 adding 'lm_eval/tasks/asdiv/README.md' 2026-05-11T15:53:30,107 adding 'lm_eval/tasks/asdiv/asdiv-cot-llama.yaml' 2026-05-11T15:53:30,109 adding 'lm_eval/tasks/asdiv/default.yaml' 2026-05-11T15:53:30,111 adding 'lm_eval/tasks/babi/README.md' 2026-05-11T15:53:30,112 adding 'lm_eval/tasks/babi/babi.yaml' 2026-05-11T15:53:30,114 adding 'lm_eval/tasks/babilong/README.md' 2026-05-11T15:53:30,115 adding 'lm_eval/tasks/babilong/_babilong_common_yaml' 2026-05-11T15:53:30,117 adding 'lm_eval/tasks/babilong/babilong.yaml' 2026-05-11T15:53:30,118 adding 'lm_eval/tasks/babilong/babilong_longctx.yaml' 2026-05-11T15:53:30,119 adding 'lm_eval/tasks/babilong/babilong_qa1.yaml' 2026-05-11T15:53:30,120 adding 'lm_eval/tasks/babilong/babilong_qa10.yaml' 2026-05-11T15:53:30,122 adding 'lm_eval/tasks/babilong/babilong_qa11.yaml' 2026-05-11T15:53:30,123 adding 'lm_eval/tasks/babilong/babilong_qa12.yaml' 2026-05-11T15:53:30,124 adding 'lm_eval/tasks/babilong/babilong_qa13.yaml' 2026-05-11T15:53:30,125 adding 'lm_eval/tasks/babilong/babilong_qa14.yaml' 2026-05-11T15:53:30,126 adding 'lm_eval/tasks/babilong/babilong_qa15.yaml' 2026-05-11T15:53:30,128 adding 'lm_eval/tasks/babilong/babilong_qa16.yaml' 2026-05-11T15:53:30,129 adding 'lm_eval/tasks/babilong/babilong_qa17.yaml' 2026-05-11T15:53:30,130 adding 'lm_eval/tasks/babilong/babilong_qa18.yaml' 2026-05-11T15:53:30,132 adding 'lm_eval/tasks/babilong/babilong_qa19.yaml' 2026-05-11T15:53:30,133 adding 'lm_eval/tasks/babilong/babilong_qa2.yaml' 2026-05-11T15:53:30,134 adding 'lm_eval/tasks/babilong/babilong_qa20.yaml' 2026-05-11T15:53:30,135 adding 'lm_eval/tasks/babilong/babilong_qa3.yaml' 2026-05-11T15:53:30,137 adding 'lm_eval/tasks/babilong/babilong_qa4.yaml' 2026-05-11T15:53:30,138 adding 'lm_eval/tasks/babilong/babilong_qa5.yaml' 2026-05-11T15:53:30,139 adding 'lm_eval/tasks/babilong/babilong_qa6.yaml' 2026-05-11T15:53:30,140 adding 'lm_eval/tasks/babilong/babilong_qa7.yaml' 2026-05-11T15:53:30,142 adding 'lm_eval/tasks/babilong/babilong_qa8.yaml' 2026-05-11T15:53:30,143 adding 'lm_eval/tasks/babilong/babilong_qa9.yaml' 2026-05-11T15:53:30,145 adding 'lm_eval/tasks/babilong/common_utils.py' 2026-05-11T15:53:30,147 adding 'lm_eval/tasks/bangla/README.md' 2026-05-11T15:53:30,148 adding 'lm_eval/tasks/bangla/bangla_boolqa.yaml' 2026-05-11T15:53:30,149 adding 'lm_eval/tasks/bangla/bangla_commonsenseqa.yaml' 2026-05-11T15:53:30,151 adding 'lm_eval/tasks/bangla/bangla_mmlu.yaml' 2026-05-11T15:53:30,152 adding 'lm_eval/tasks/bangla/bangla_openbookqa.yaml' 2026-05-11T15:53:30,153 adding 'lm_eval/tasks/bangla/bangla_piqa.yaml' 2026-05-11T15:53:30,155 adding 'lm_eval/tasks/basque_bench/README.md' 2026-05-11T15:53:30,157 adding 'lm_eval/tasks/basque_bench/arc_eu_challenge.yaml' 2026-05-11T15:53:30,158 adding 'lm_eval/tasks/basque_bench/arc_eu_easy.yaml' 2026-05-11T15:53:30,159 adding 'lm_eval/tasks/basque_bench/basque_bench.yaml' 2026-05-11T15:53:30,160 adding 'lm_eval/tasks/basque_bench/mgsm_direct_eu.yaml' 2026-05-11T15:53:30,162 adding 'lm_eval/tasks/basque_bench/mgsm_native_cot_eu.yaml' 2026-05-11T15:53:30,163 adding 'lm_eval/tasks/basque_bench/paws_eu.yaml' 2026-05-11T15:53:30,164 adding 'lm_eval/tasks/basque_bench/piqa_eu.yaml' 2026-05-11T15:53:30,165 adding 'lm_eval/tasks/basque_bench/utils.py' 2026-05-11T15:53:30,166 adding 'lm_eval/tasks/basque_bench/wnli_eu.yaml' 2026-05-11T15:53:30,168 adding 'lm_eval/tasks/basque_bench/xcopa_eu.yaml' 2026-05-11T15:53:30,170 adding 'lm_eval/tasks/basque_bench/flores_eu/_flores_common_yaml' 2026-05-11T15:53:30,171 adding 'lm_eval/tasks/basque_bench/flores_eu/create_yamls_flores_eu.py' 2026-05-11T15:53:30,172 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_ca-eu.yaml' 2026-05-11T15:53:30,174 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_de-eu.yaml' 2026-05-11T15:53:30,175 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_en-eu.yaml' 2026-05-11T15:53:30,176 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_es-eu.yaml' 2026-05-11T15:53:30,177 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-ca.yaml' 2026-05-11T15:53:30,178 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-de.yaml' 2026-05-11T15:53:30,179 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-en.yaml' 2026-05-11T15:53:30,181 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-es.yaml' 2026-05-11T15:53:30,182 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-fr.yaml' 2026-05-11T15:53:30,183 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-gl.yaml' 2026-05-11T15:53:30,184 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-it.yaml' 2026-05-11T15:53:30,185 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu-pt.yaml' 2026-05-11T15:53:30,187 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_eu.yaml' 2026-05-11T15:53:30,188 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_fr-eu.yaml' 2026-05-11T15:53:30,189 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_gl-eu.yaml' 2026-05-11T15:53:30,190 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_it-eu.yaml' 2026-05-11T15:53:30,191 adding 'lm_eval/tasks/basque_bench/flores_eu/flores_pt-eu.yaml' 2026-05-11T15:53:30,194 adding 'lm_eval/tasks/basqueglue/README.md' 2026-05-11T15:53:30,195 adding 'lm_eval/tasks/basqueglue/bec.yaml' 2026-05-11T15:53:30,196 adding 'lm_eval/tasks/basqueglue/bhtc.yaml' 2026-05-11T15:53:30,197 adding 'lm_eval/tasks/basqueglue/coref.yaml' 2026-05-11T15:53:30,198 adding 'lm_eval/tasks/basqueglue/qnli.yaml' 2026-05-11T15:53:30,200 adding 'lm_eval/tasks/basqueglue/utils.py' 2026-05-11T15:53:30,201 adding 'lm_eval/tasks/basqueglue/vaxx.yaml' 2026-05-11T15:53:30,202 adding 'lm_eval/tasks/basqueglue/wic.yaml' 2026-05-11T15:53:30,204 adding 'lm_eval/tasks/bbh/README.md' 2026-05-11T15:53:30,205 adding 'lm_eval/tasks/bbh/_generate_configs.py' 2026-05-11T15:53:30,208 adding 'lm_eval/tasks/bbh/cot_fewshot/_bbh.yaml' 2026-05-11T15:53:30,209 adding 'lm_eval/tasks/bbh/cot_fewshot/_bbh_cot_fewshot.yaml' 2026-05-11T15:53:30,210 adding 'lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml' 2026-05-11T15:53:30,212 adding 'lm_eval/tasks/bbh/cot_fewshot/boolean_expressions.yaml' 2026-05-11T15:53:30,213 adding 'lm_eval/tasks/bbh/cot_fewshot/causal_judgement.yaml' 2026-05-11T15:53:30,215 adding 'lm_eval/tasks/bbh/cot_fewshot/date_understanding.yaml' 2026-05-11T15:53:30,216 adding 'lm_eval/tasks/bbh/cot_fewshot/disambiguation_qa.yaml' 2026-05-11T15:53:30,217 adding 'lm_eval/tasks/bbh/cot_fewshot/dyck_languages.yaml' 2026-05-11T15:53:30,219 adding 'lm_eval/tasks/bbh/cot_fewshot/formal_fallacies.yaml' 2026-05-11T15:53:30,220 adding 'lm_eval/tasks/bbh/cot_fewshot/geometric_shapes.yaml' 2026-05-11T15:53:30,222 adding 'lm_eval/tasks/bbh/cot_fewshot/hyperbaton.yaml' 2026-05-11T15:53:30,223 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_five_objects.yaml' 2026-05-11T15:53:30,224 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_seven_objects.yaml' 2026-05-11T15:53:30,226 adding 'lm_eval/tasks/bbh/cot_fewshot/logical_deduction_three_objects.yaml' 2026-05-11T15:53:30,227 adding 'lm_eval/tasks/bbh/cot_fewshot/movie_recommendation.yaml' 2026-05-11T15:53:30,229 adding 'lm_eval/tasks/bbh/cot_fewshot/multistep_arithmetic_two.yaml' 2026-05-11T15:53:30,230 adding 'lm_eval/tasks/bbh/cot_fewshot/navigate.yaml' 2026-05-11T15:53:30,231 adding 'lm_eval/tasks/bbh/cot_fewshot/object_counting.yaml' 2026-05-11T15:53:30,233 adding 'lm_eval/tasks/bbh/cot_fewshot/penguins_in_a_table.yaml' 2026-05-11T15:53:30,235 adding 'lm_eval/tasks/bbh/cot_fewshot/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:30,236 adding 'lm_eval/tasks/bbh/cot_fewshot/ruin_names.yaml' 2026-05-11T15:53:30,238 adding 'lm_eval/tasks/bbh/cot_fewshot/salient_translation_error_detection.yaml' 2026-05-11T15:53:30,239 adding 'lm_eval/tasks/bbh/cot_fewshot/snarks.yaml' 2026-05-11T15:53:30,241 adding 'lm_eval/tasks/bbh/cot_fewshot/sports_understanding.yaml' 2026-05-11T15:53:30,242 adding 'lm_eval/tasks/bbh/cot_fewshot/temporal_sequences.yaml' 2026-05-11T15:53:30,243 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T15:53:30,245 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T15:53:30,247 adding 'lm_eval/tasks/bbh/cot_fewshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T15:53:30,248 adding 'lm_eval/tasks/bbh/cot_fewshot/web_of_lies.yaml' 2026-05-11T15:53:30,250 adding 'lm_eval/tasks/bbh/cot_fewshot/word_sorting.yaml' 2026-05-11T15:53:30,252 adding 'lm_eval/tasks/bbh/cot_zeroshot/_bbh_cot_zeroshot.yaml' 2026-05-11T15:53:30,253 adding 'lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml' 2026-05-11T15:53:30,254 adding 'lm_eval/tasks/bbh/cot_zeroshot/boolean_expressions.yaml' 2026-05-11T15:53:30,255 adding 'lm_eval/tasks/bbh/cot_zeroshot/causal_judgement.yaml' 2026-05-11T15:53:30,257 adding 'lm_eval/tasks/bbh/cot_zeroshot/date_understanding.yaml' 2026-05-11T15:53:30,258 adding 'lm_eval/tasks/bbh/cot_zeroshot/disambiguation_qa.yaml' 2026-05-11T15:53:30,259 adding 'lm_eval/tasks/bbh/cot_zeroshot/dyck_languages.yaml' 2026-05-11T15:53:30,260 adding 'lm_eval/tasks/bbh/cot_zeroshot/formal_fallacies.yaml' 2026-05-11T15:53:30,262 adding 'lm_eval/tasks/bbh/cot_zeroshot/geometric_shapes.yaml' 2026-05-11T15:53:30,263 adding 'lm_eval/tasks/bbh/cot_zeroshot/hyperbaton.yaml' 2026-05-11T15:53:30,264 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_five_objects.yaml' 2026-05-11T15:53:30,265 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_seven_objects.yaml' 2026-05-11T15:53:30,267 adding 'lm_eval/tasks/bbh/cot_zeroshot/logical_deduction_three_objects.yaml' 2026-05-11T15:53:30,268 adding 'lm_eval/tasks/bbh/cot_zeroshot/movie_recommendation.yaml' 2026-05-11T15:53:30,269 adding 'lm_eval/tasks/bbh/cot_zeroshot/multistep_arithmetic_two.yaml' 2026-05-11T15:53:30,270 adding 'lm_eval/tasks/bbh/cot_zeroshot/navigate.yaml' 2026-05-11T15:53:30,272 adding 'lm_eval/tasks/bbh/cot_zeroshot/object_counting.yaml' 2026-05-11T15:53:30,273 adding 'lm_eval/tasks/bbh/cot_zeroshot/penguins_in_a_table.yaml' 2026-05-11T15:53:30,274 adding 'lm_eval/tasks/bbh/cot_zeroshot/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:30,275 adding 'lm_eval/tasks/bbh/cot_zeroshot/ruin_names.yaml' 2026-05-11T15:53:30,277 adding 'lm_eval/tasks/bbh/cot_zeroshot/salient_translation_error_detection.yaml' 2026-05-11T15:53:30,278 adding 'lm_eval/tasks/bbh/cot_zeroshot/snarks.yaml' 2026-05-11T15:53:30,279 adding 'lm_eval/tasks/bbh/cot_zeroshot/sports_understanding.yaml' 2026-05-11T15:53:30,281 adding 'lm_eval/tasks/bbh/cot_zeroshot/temporal_sequences.yaml' 2026-05-11T15:53:30,282 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T15:53:30,283 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T15:53:30,284 adding 'lm_eval/tasks/bbh/cot_zeroshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T15:53:30,286 adding 'lm_eval/tasks/bbh/cot_zeroshot/utils.py' 2026-05-11T15:53:30,287 adding 'lm_eval/tasks/bbh/cot_zeroshot/web_of_lies.yaml' 2026-05-11T15:53:30,288 adding 'lm_eval/tasks/bbh/cot_zeroshot/word_sorting.yaml' 2026-05-11T15:53:30,291 adding 'lm_eval/tasks/bbh/fewshot/_bbh_fewshot.yaml' 2026-05-11T15:53:30,292 adding 'lm_eval/tasks/bbh/fewshot/_fewshot_template_yaml' 2026-05-11T15:53:30,293 adding 'lm_eval/tasks/bbh/fewshot/boolean_expressions.yaml' 2026-05-11T15:53:30,294 adding 'lm_eval/tasks/bbh/fewshot/causal_judgement.yaml' 2026-05-11T15:53:30,296 adding 'lm_eval/tasks/bbh/fewshot/date_understanding.yaml' 2026-05-11T15:53:30,297 adding 'lm_eval/tasks/bbh/fewshot/disambiguation_qa.yaml' 2026-05-11T15:53:30,298 adding 'lm_eval/tasks/bbh/fewshot/dyck_languages.yaml' 2026-05-11T15:53:30,299 adding 'lm_eval/tasks/bbh/fewshot/formal_fallacies.yaml' 2026-05-11T15:53:30,300 adding 'lm_eval/tasks/bbh/fewshot/geometric_shapes.yaml' 2026-05-11T15:53:30,302 adding 'lm_eval/tasks/bbh/fewshot/hyperbaton.yaml' 2026-05-11T15:53:30,303 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_five_objects.yaml' 2026-05-11T15:53:30,304 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_seven_objects.yaml' 2026-05-11T15:53:30,305 adding 'lm_eval/tasks/bbh/fewshot/logical_deduction_three_objects.yaml' 2026-05-11T15:53:30,307 adding 'lm_eval/tasks/bbh/fewshot/movie_recommendation.yaml' 2026-05-11T15:53:30,308 adding 'lm_eval/tasks/bbh/fewshot/multistep_arithmetic_two.yaml' 2026-05-11T15:53:30,309 adding 'lm_eval/tasks/bbh/fewshot/navigate.yaml' 2026-05-11T15:53:30,310 adding 'lm_eval/tasks/bbh/fewshot/object_counting.yaml' 2026-05-11T15:53:30,311 adding 'lm_eval/tasks/bbh/fewshot/penguins_in_a_table.yaml' 2026-05-11T15:53:30,313 adding 'lm_eval/tasks/bbh/fewshot/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:30,314 adding 'lm_eval/tasks/bbh/fewshot/ruin_names.yaml' 2026-05-11T15:53:30,315 adding 'lm_eval/tasks/bbh/fewshot/salient_translation_error_detection.yaml' 2026-05-11T15:53:30,316 adding 'lm_eval/tasks/bbh/fewshot/snarks.yaml' 2026-05-11T15:53:30,318 adding 'lm_eval/tasks/bbh/fewshot/sports_understanding.yaml' 2026-05-11T15:53:30,319 adding 'lm_eval/tasks/bbh/fewshot/temporal_sequences.yaml' 2026-05-11T15:53:30,320 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T15:53:30,322 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T15:53:30,323 adding 'lm_eval/tasks/bbh/fewshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T15:53:30,324 adding 'lm_eval/tasks/bbh/fewshot/web_of_lies.yaml' 2026-05-11T15:53:30,325 adding 'lm_eval/tasks/bbh/fewshot/word_sorting.yaml' 2026-05-11T15:53:30,328 adding 'lm_eval/tasks/bbh/zeroshot/_bbh_zeroshot.yaml' 2026-05-11T15:53:30,329 adding 'lm_eval/tasks/bbh/zeroshot/_zeroshot_template_yaml' 2026-05-11T15:53:30,330 adding 'lm_eval/tasks/bbh/zeroshot/boolean_expressions.yaml' 2026-05-11T15:53:30,332 adding 'lm_eval/tasks/bbh/zeroshot/causal_judgement.yaml' 2026-05-11T15:53:30,333 adding 'lm_eval/tasks/bbh/zeroshot/date_understanding.yaml' 2026-05-11T15:53:30,334 adding 'lm_eval/tasks/bbh/zeroshot/disambiguation_qa.yaml' 2026-05-11T15:53:30,335 adding 'lm_eval/tasks/bbh/zeroshot/dyck_languages.yaml' 2026-05-11T15:53:30,337 adding 'lm_eval/tasks/bbh/zeroshot/formal_fallacies.yaml' 2026-05-11T15:53:30,338 adding 'lm_eval/tasks/bbh/zeroshot/geometric_shapes.yaml' 2026-05-11T15:53:30,339 adding 'lm_eval/tasks/bbh/zeroshot/hyperbaton.yaml' 2026-05-11T15:53:30,341 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_five_objects.yaml' 2026-05-11T15:53:30,342 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_seven_objects.yaml' 2026-05-11T15:53:30,343 adding 'lm_eval/tasks/bbh/zeroshot/logical_deduction_three_objects.yaml' 2026-05-11T15:53:30,344 adding 'lm_eval/tasks/bbh/zeroshot/movie_recommendation.yaml' 2026-05-11T15:53:30,345 adding 'lm_eval/tasks/bbh/zeroshot/multistep_arithmetic_two.yaml' 2026-05-11T15:53:30,347 adding 'lm_eval/tasks/bbh/zeroshot/navigate.yaml' 2026-05-11T15:53:30,348 adding 'lm_eval/tasks/bbh/zeroshot/object_counting.yaml' 2026-05-11T15:53:30,349 adding 'lm_eval/tasks/bbh/zeroshot/penguins_in_a_table.yaml' 2026-05-11T15:53:30,351 adding 'lm_eval/tasks/bbh/zeroshot/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:30,352 adding 'lm_eval/tasks/bbh/zeroshot/ruin_names.yaml' 2026-05-11T15:53:30,353 adding 'lm_eval/tasks/bbh/zeroshot/salient_translation_error_detection.yaml' 2026-05-11T15:53:30,354 adding 'lm_eval/tasks/bbh/zeroshot/snarks.yaml' 2026-05-11T15:53:30,355 adding 'lm_eval/tasks/bbh/zeroshot/sports_understanding.yaml' 2026-05-11T15:53:30,356 adding 'lm_eval/tasks/bbh/zeroshot/temporal_sequences.yaml' 2026-05-11T15:53:30,358 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T15:53:30,359 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T15:53:30,360 adding 'lm_eval/tasks/bbh/zeroshot/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T15:53:30,362 adding 'lm_eval/tasks/bbh/zeroshot/utils.py' 2026-05-11T15:53:30,363 adding 'lm_eval/tasks/bbh/zeroshot/web_of_lies.yaml' 2026-05-11T15:53:30,364 adding 'lm_eval/tasks/bbh/zeroshot/word_sorting.yaml' 2026-05-11T15:53:30,366 adding 'lm_eval/tasks/bbq/README.md' 2026-05-11T15:53:30,368 adding 'lm_eval/tasks/bbq/bbq_generate.yaml' 2026-05-11T15:53:30,369 adding 'lm_eval/tasks/bbq/bbq_generate_ambig.yaml' 2026-05-11T15:53:30,370 adding 'lm_eval/tasks/bbq/bbq_generate_disambig.yaml' 2026-05-11T15:53:30,371 adding 'lm_eval/tasks/bbq/bbq_multiple_choice.yaml' 2026-05-11T15:53:30,372 adding 'lm_eval/tasks/bbq/bbq_multiple_choice_ambig.yaml' 2026-05-11T15:53:30,373 adding 'lm_eval/tasks/bbq/bbq_multiple_choice_disambig.yaml' 2026-05-11T15:53:30,375 adding 'lm_eval/tasks/bbq/utils.py' 2026-05-11T15:53:30,377 adding 'lm_eval/tasks/bear/README.md' 2026-05-11T15:53:30,379 adding 'lm_eval/tasks/bear/bear.yaml' 2026-05-11T15:53:30,380 adding 'lm_eval/tasks/bear/bear_big.yaml' 2026-05-11T15:53:30,384 adding 'lm_eval/tasks/belebele/README.md' 2026-05-11T15:53:30,385 adding 'lm_eval/tasks/belebele/_belebele.yaml' 2026-05-11T15:53:30,387 adding 'lm_eval/tasks/belebele/_default_template_yaml' 2026-05-11T15:53:30,388 adding 'lm_eval/tasks/belebele/_generate_configs.py' 2026-05-11T15:53:30,389 adding 'lm_eval/tasks/belebele/belebele_acm_Arab.yaml' 2026-05-11T15:53:30,391 adding 'lm_eval/tasks/belebele/belebele_afr_Latn.yaml' 2026-05-11T15:53:30,392 adding 'lm_eval/tasks/belebele/belebele_als_Latn.yaml' 2026-05-11T15:53:30,393 adding 'lm_eval/tasks/belebele/belebele_amh_Ethi.yaml' 2026-05-11T15:53:30,394 adding 'lm_eval/tasks/belebele/belebele_apc_Arab.yaml' 2026-05-11T15:53:30,395 adding 'lm_eval/tasks/belebele/belebele_arb_Arab.yaml' 2026-05-11T15:53:30,396 adding 'lm_eval/tasks/belebele/belebele_arb_Latn.yaml' 2026-05-11T15:53:30,398 adding 'lm_eval/tasks/belebele/belebele_ars_Arab.yaml' 2026-05-11T15:53:30,399 adding 'lm_eval/tasks/belebele/belebele_ary_Arab.yaml' 2026-05-11T15:53:30,400 adding 'lm_eval/tasks/belebele/belebele_arz_Arab.yaml' 2026-05-11T15:53:30,401 adding 'lm_eval/tasks/belebele/belebele_asm_Beng.yaml' 2026-05-11T15:53:30,403 adding 'lm_eval/tasks/belebele/belebele_azj_Latn.yaml' 2026-05-11T15:53:30,404 adding 'lm_eval/tasks/belebele/belebele_bam_Latn.yaml' 2026-05-11T15:53:30,405 adding 'lm_eval/tasks/belebele/belebele_ben_Beng.yaml' 2026-05-11T15:53:30,406 adding 'lm_eval/tasks/belebele/belebele_ben_Latn.yaml' 2026-05-11T15:53:30,407 adding 'lm_eval/tasks/belebele/belebele_bod_Tibt.yaml' 2026-05-11T15:53:30,408 adding 'lm_eval/tasks/belebele/belebele_bul_Cyrl.yaml' 2026-05-11T15:53:30,410 adding 'lm_eval/tasks/belebele/belebele_cat_Latn.yaml' 2026-05-11T15:53:30,411 adding 'lm_eval/tasks/belebele/belebele_ceb_Latn.yaml' 2026-05-11T15:53:30,412 adding 'lm_eval/tasks/belebele/belebele_ces_Latn.yaml' 2026-05-11T15:53:30,413 adding 'lm_eval/tasks/belebele/belebele_ckb_Arab.yaml' 2026-05-11T15:53:30,414 adding 'lm_eval/tasks/belebele/belebele_dan_Latn.yaml' 2026-05-11T15:53:30,415 adding 'lm_eval/tasks/belebele/belebele_deu_Latn.yaml' 2026-05-11T15:53:30,417 adding 'lm_eval/tasks/belebele/belebele_ell_Grek.yaml' 2026-05-11T15:53:30,418 adding 'lm_eval/tasks/belebele/belebele_eng_Latn.yaml' 2026-05-11T15:53:30,419 adding 'lm_eval/tasks/belebele/belebele_est_Latn.yaml' 2026-05-11T15:53:30,420 adding 'lm_eval/tasks/belebele/belebele_eus_Latn.yaml' 2026-05-11T15:53:30,421 adding 'lm_eval/tasks/belebele/belebele_fin_Latn.yaml' 2026-05-11T15:53:30,422 adding 'lm_eval/tasks/belebele/belebele_fra_Latn.yaml' 2026-05-11T15:53:30,423 adding 'lm_eval/tasks/belebele/belebele_fuv_Latn.yaml' 2026-05-11T15:53:30,424 adding 'lm_eval/tasks/belebele/belebele_gaz_Latn.yaml' 2026-05-11T15:53:30,425 adding 'lm_eval/tasks/belebele/belebele_grn_Latn.yaml' 2026-05-11T15:53:30,426 adding 'lm_eval/tasks/belebele/belebele_guj_Gujr.yaml' 2026-05-11T15:53:30,427 adding 'lm_eval/tasks/belebele/belebele_hat_Latn.yaml' 2026-05-11T15:53:30,428 adding 'lm_eval/tasks/belebele/belebele_hau_Latn.yaml' 2026-05-11T15:53:30,429 adding 'lm_eval/tasks/belebele/belebele_heb_Hebr.yaml' 2026-05-11T15:53:30,431 adding 'lm_eval/tasks/belebele/belebele_hin_Deva.yaml' 2026-05-11T15:53:30,432 adding 'lm_eval/tasks/belebele/belebele_hin_Latn.yaml' 2026-05-11T15:53:30,433 adding 'lm_eval/tasks/belebele/belebele_hrv_Latn.yaml' 2026-05-11T15:53:30,436 adding 'lm_eval/tasks/belebele/belebele_hun_Latn.yaml' 2026-05-11T15:53:30,437 adding 'lm_eval/tasks/belebele/belebele_hye_Armn.yaml' 2026-05-11T15:53:30,438 adding 'lm_eval/tasks/belebele/belebele_ibo_Latn.yaml' 2026-05-11T15:53:30,440 adding 'lm_eval/tasks/belebele/belebele_ilo_Latn.yaml' 2026-05-11T15:53:30,441 adding 'lm_eval/tasks/belebele/belebele_ind_Latn.yaml' 2026-05-11T15:53:30,442 adding 'lm_eval/tasks/belebele/belebele_isl_Latn.yaml' 2026-05-11T15:53:30,443 adding 'lm_eval/tasks/belebele/belebele_ita_Latn.yaml' 2026-05-11T15:53:30,444 adding 'lm_eval/tasks/belebele/belebele_jav_Latn.yaml' 2026-05-11T15:53:30,445 adding 'lm_eval/tasks/belebele/belebele_jpn_Jpan.yaml' 2026-05-11T15:53:30,446 adding 'lm_eval/tasks/belebele/belebele_kac_Latn.yaml' 2026-05-11T15:53:30,447 adding 'lm_eval/tasks/belebele/belebele_kan_Knda.yaml' 2026-05-11T15:53:30,449 adding 'lm_eval/tasks/belebele/belebele_kat_Geor.yaml' 2026-05-11T15:53:30,450 adding 'lm_eval/tasks/belebele/belebele_kaz_Cyrl.yaml' 2026-05-11T15:53:30,451 adding 'lm_eval/tasks/belebele/belebele_kea_Latn.yaml' 2026-05-11T15:53:30,452 adding 'lm_eval/tasks/belebele/belebele_khk_Cyrl.yaml' 2026-05-11T15:53:30,453 adding 'lm_eval/tasks/belebele/belebele_khm_Khmr.yaml' 2026-05-11T15:53:30,454 adding 'lm_eval/tasks/belebele/belebele_kin_Latn.yaml' 2026-05-11T15:53:30,455 adding 'lm_eval/tasks/belebele/belebele_kir_Cyrl.yaml' 2026-05-11T15:53:30,457 adding 'lm_eval/tasks/belebele/belebele_kor_Hang.yaml' 2026-05-11T15:53:30,458 adding 'lm_eval/tasks/belebele/belebele_lao_Laoo.yaml' 2026-05-11T15:53:30,459 adding 'lm_eval/tasks/belebele/belebele_lin_Latn.yaml' 2026-05-11T15:53:30,460 adding 'lm_eval/tasks/belebele/belebele_lit_Latn.yaml' 2026-05-11T15:53:30,461 adding 'lm_eval/tasks/belebele/belebele_lug_Latn.yaml' 2026-05-11T15:53:30,462 adding 'lm_eval/tasks/belebele/belebele_luo_Latn.yaml' 2026-05-11T15:53:30,464 adding 'lm_eval/tasks/belebele/belebele_lvs_Latn.yaml' 2026-05-11T15:53:30,465 adding 'lm_eval/tasks/belebele/belebele_mal_Mlym.yaml' 2026-05-11T15:53:30,466 adding 'lm_eval/tasks/belebele/belebele_mar_Deva.yaml' 2026-05-11T15:53:30,467 adding 'lm_eval/tasks/belebele/belebele_mkd_Cyrl.yaml' 2026-05-11T15:53:30,469 adding 'lm_eval/tasks/belebele/belebele_mlt_Latn.yaml' 2026-05-11T15:53:30,470 adding 'lm_eval/tasks/belebele/belebele_mri_Latn.yaml' 2026-05-11T15:53:30,471 adding 'lm_eval/tasks/belebele/belebele_mya_Mymr.yaml' 2026-05-11T15:53:30,472 adding 'lm_eval/tasks/belebele/belebele_nld_Latn.yaml' 2026-05-11T15:53:30,473 adding 'lm_eval/tasks/belebele/belebele_nob_Latn.yaml' 2026-05-11T15:53:30,474 adding 'lm_eval/tasks/belebele/belebele_npi_Deva.yaml' 2026-05-11T15:53:30,475 adding 'lm_eval/tasks/belebele/belebele_npi_Latn.yaml' 2026-05-11T15:53:30,477 adding 'lm_eval/tasks/belebele/belebele_nso_Latn.yaml' 2026-05-11T15:53:30,478 adding 'lm_eval/tasks/belebele/belebele_nya_Latn.yaml' 2026-05-11T15:53:30,479 adding 'lm_eval/tasks/belebele/belebele_ory_Orya.yaml' 2026-05-11T15:53:30,480 adding 'lm_eval/tasks/belebele/belebele_pan_Guru.yaml' 2026-05-11T15:53:30,481 adding 'lm_eval/tasks/belebele/belebele_pbt_Arab.yaml' 2026-05-11T15:53:30,482 adding 'lm_eval/tasks/belebele/belebele_pes_Arab.yaml' 2026-05-11T15:53:30,483 adding 'lm_eval/tasks/belebele/belebele_plt_Latn.yaml' 2026-05-11T15:53:30,484 adding 'lm_eval/tasks/belebele/belebele_pol_Latn.yaml' 2026-05-11T15:53:30,485 adding 'lm_eval/tasks/belebele/belebele_por_Latn.yaml' 2026-05-11T15:53:30,486 adding 'lm_eval/tasks/belebele/belebele_ron_Latn.yaml' 2026-05-11T15:53:30,487 adding 'lm_eval/tasks/belebele/belebele_rus_Cyrl.yaml' 2026-05-11T15:53:30,488 adding 'lm_eval/tasks/belebele/belebele_shn_Mymr.yaml' 2026-05-11T15:53:30,489 adding 'lm_eval/tasks/belebele/belebele_sin_Latn.yaml' 2026-05-11T15:53:30,491 adding 'lm_eval/tasks/belebele/belebele_sin_Sinh.yaml' 2026-05-11T15:53:30,492 adding 'lm_eval/tasks/belebele/belebele_slk_Latn.yaml' 2026-05-11T15:53:30,493 adding 'lm_eval/tasks/belebele/belebele_slv_Latn.yaml' 2026-05-11T15:53:30,494 adding 'lm_eval/tasks/belebele/belebele_sna_Latn.yaml' 2026-05-11T15:53:30,495 adding 'lm_eval/tasks/belebele/belebele_snd_Arab.yaml' 2026-05-11T15:53:30,496 adding 'lm_eval/tasks/belebele/belebele_som_Latn.yaml' 2026-05-11T15:53:30,497 adding 'lm_eval/tasks/belebele/belebele_sot_Latn.yaml' 2026-05-11T15:53:30,498 adding 'lm_eval/tasks/belebele/belebele_spa_Latn.yaml' 2026-05-11T15:53:30,500 adding 'lm_eval/tasks/belebele/belebele_srp_Cyrl.yaml' 2026-05-11T15:53:30,501 adding 'lm_eval/tasks/belebele/belebele_ssw_Latn.yaml' 2026-05-11T15:53:30,502 adding 'lm_eval/tasks/belebele/belebele_sun_Latn.yaml' 2026-05-11T15:53:30,503 adding 'lm_eval/tasks/belebele/belebele_swe_Latn.yaml' 2026-05-11T15:53:30,504 adding 'lm_eval/tasks/belebele/belebele_swh_Latn.yaml' 2026-05-11T15:53:30,505 adding 'lm_eval/tasks/belebele/belebele_tam_Taml.yaml' 2026-05-11T15:53:30,506 adding 'lm_eval/tasks/belebele/belebele_tel_Telu.yaml' 2026-05-11T15:53:30,507 adding 'lm_eval/tasks/belebele/belebele_tgk_Cyrl.yaml' 2026-05-11T15:53:30,509 adding 'lm_eval/tasks/belebele/belebele_tgl_Latn.yaml' 2026-05-11T15:53:30,510 adding 'lm_eval/tasks/belebele/belebele_tha_Thai.yaml' 2026-05-11T15:53:30,511 adding 'lm_eval/tasks/belebele/belebele_tir_Ethi.yaml' 2026-05-11T15:53:30,512 adding 'lm_eval/tasks/belebele/belebele_tsn_Latn.yaml' 2026-05-11T15:53:30,513 adding 'lm_eval/tasks/belebele/belebele_tso_Latn.yaml' 2026-05-11T15:53:30,514 adding 'lm_eval/tasks/belebele/belebele_tur_Latn.yaml' 2026-05-11T15:53:30,515 adding 'lm_eval/tasks/belebele/belebele_ukr_Cyrl.yaml' 2026-05-11T15:53:30,517 adding 'lm_eval/tasks/belebele/belebele_urd_Arab.yaml' 2026-05-11T15:53:30,518 adding 'lm_eval/tasks/belebele/belebele_urd_Latn.yaml' 2026-05-11T15:53:30,519 adding 'lm_eval/tasks/belebele/belebele_uzn_Latn.yaml' 2026-05-11T15:53:30,520 adding 'lm_eval/tasks/belebele/belebele_vie_Latn.yaml' 2026-05-11T15:53:30,521 adding 'lm_eval/tasks/belebele/belebele_war_Latn.yaml' 2026-05-11T15:53:30,522 adding 'lm_eval/tasks/belebele/belebele_wol_Latn.yaml' 2026-05-11T15:53:30,524 adding 'lm_eval/tasks/belebele/belebele_xho_Latn.yaml' 2026-05-11T15:53:30,525 adding 'lm_eval/tasks/belebele/belebele_yor_Latn.yaml' 2026-05-11T15:53:30,526 adding 'lm_eval/tasks/belebele/belebele_zho_Hans.yaml' 2026-05-11T15:53:30,527 adding 'lm_eval/tasks/belebele/belebele_zho_Hant.yaml' 2026-05-11T15:53:30,528 adding 'lm_eval/tasks/belebele/belebele_zsm_Latn.yaml' 2026-05-11T15:53:30,529 adding 'lm_eval/tasks/belebele/belebele_zul_Latn.yaml' 2026-05-11T15:53:30,531 adding 'lm_eval/tasks/benchmarks/README.md' 2026-05-11T15:53:30,532 adding 'lm_eval/tasks/benchmarks/minerva_math.yaml' 2026-05-11T15:53:30,534 adding 'lm_eval/tasks/benchmarks/openllm.yaml' 2026-05-11T15:53:30,535 adding 'lm_eval/tasks/benchmarks/pythia.yaml' 2026-05-11T15:53:30,536 adding 'lm_eval/tasks/benchmarks/t0_eval.yaml' 2026-05-11T15:53:30,538 adding 'lm_eval/tasks/benchmarks/flan/_held_in_template_yaml' 2026-05-11T15:53:30,540 adding 'lm_eval/tasks/benchmarks/flan/flan_held_in.yaml' 2026-05-11T15:53:30,541 adding 'lm_eval/tasks/benchmarks/flan/flan_held_out.yaml' 2026-05-11T15:53:30,543 adding 'lm_eval/tasks/benchmarks/multimedqa/README.md' 2026-05-11T15:53:30,544 adding 'lm_eval/tasks/benchmarks/multimedqa/multimedqa.yaml' 2026-05-11T15:53:30,546 adding 'lm_eval/tasks/bertaqa/README.md' 2026-05-11T15:53:30,547 adding 'lm_eval/tasks/bertaqa/_bertaqa_template' 2026-05-11T15:53:30,548 adding 'lm_eval/tasks/bertaqa/bertaqa_en.yaml' 2026-05-11T15:53:30,549 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_gemma-7b.yaml' 2026-05-11T15:53:30,550 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_hitz.yaml' 2026-05-11T15:53:30,552 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_itzuli.yaml' 2026-05-11T15:53:30,553 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.1.yaml' 2026-05-11T15:53:30,554 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-13b-v1.yaml' 2026-05-11T15:53:30,555 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.1.yaml' 2026-05-11T15:53:30,556 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-70b-v1.yaml' 2026-05-11T15:53:30,558 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.1.yaml' 2026-05-11T15:53:30,559 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_latxa-7b-v1.yaml' 2026-05-11T15:53:30,560 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-13b.yaml' 2026-05-11T15:53:30,561 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-70b.yaml' 2026-05-11T15:53:30,562 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_llama-2-7b.yaml' 2026-05-11T15:53:30,563 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_madlad.yaml' 2026-05-11T15:53:30,564 adding 'lm_eval/tasks/bertaqa/bertaqa_en_mt_nllb.yaml' 2026-05-11T15:53:30,566 adding 'lm_eval/tasks/bertaqa/bertaqa_eu.yaml' 2026-05-11T15:53:30,568 adding 'lm_eval/tasks/bhs/README.md' 2026-05-11T15:53:30,569 adding 'lm_eval/tasks/bhs/_template_yaml' 2026-05-11T15:53:30,571 adding 'lm_eval/tasks/bhs/basque-DO-S_DO_V_AUX.yaml' 2026-05-11T15:53:30,572 adding 'lm_eval/tasks/bhs/basque-DO-S_IO_DO_V_AUX.yaml' 2026-05-11T15:53:30,573 adding 'lm_eval/tasks/bhs/basque-IO-IO_S_V_AUX.yaml' 2026-05-11T15:53:30,574 adding 'lm_eval/tasks/bhs/basque-IO-S_IO_DO_V_AUX.yaml' 2026-05-11T15:53:30,575 adding 'lm_eval/tasks/bhs/basque-S-IO_S_V_AUX.yaml' 2026-05-11T15:53:30,576 adding 'lm_eval/tasks/bhs/basque-S-S_DO_V_AUX.yaml' 2026-05-11T15:53:30,577 adding 'lm_eval/tasks/bhs/basque-S-S_IO_DO_V_AUX.yaml' 2026-05-11T15:53:30,579 adding 'lm_eval/tasks/bhs/basque-S-S_V_AUX.yaml' 2026-05-11T15:53:30,580 adding 'lm_eval/tasks/bhs/bhs_basque.yaml' 2026-05-11T15:53:30,581 adding 'lm_eval/tasks/bhs/bhs_hindi.yaml' 2026-05-11T15:53:30,582 adding 'lm_eval/tasks/bhs/bhs_swahili.yaml' 2026-05-11T15:53:30,583 adding 'lm_eval/tasks/bhs/hindi-S_O_V.yaml' 2026-05-11T15:53:30,584 adding 'lm_eval/tasks/bhs/hindi-S_PossPRN_O_V.yaml' 2026-05-11T15:53:30,586 adding 'lm_eval/tasks/bhs/hindi-S_PossPRN_PossN_O_V.yaml' 2026-05-11T15:53:30,587 adding 'lm_eval/tasks/bhs/hindi-S_ne_O_V.yaml' 2026-05-11T15:53:30,588 adding 'lm_eval/tasks/bhs/hindi-S_ne_PossPRN_O_V.yaml' 2026-05-11T15:53:30,589 adding 'lm_eval/tasks/bhs/hindi-S_ne_PossPRN_PossN_O_V.yaml' 2026-05-11T15:53:30,590 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_V_ni_AN.yaml' 2026-05-11T15:53:30,592 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_AP_ni_AN.yaml' 2026-05-11T15:53:30,593 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V.yaml' 2026-05-11T15:53:30,594 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_A_V1_V2.yaml' 2026-05-11T15:53:30,595 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_V.yaml' 2026-05-11T15:53:30,596 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_D_ni_A.yaml' 2026-05-11T15:53:30,597 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_V.yaml' 2026-05-11T15:53:30,599 adding 'lm_eval/tasks/bhs/swahili-N_of_Poss_ni_A.yaml' 2026-05-11T15:53:30,601 adding 'lm_eval/tasks/bigbench/README.md' 2026-05-11T15:53:30,603 adding 'lm_eval/tasks/bigbench/generate_tasks.py' 2026-05-11T15:53:30,604 adding 'lm_eval/tasks/bigbench/generate_until_template_yaml' 2026-05-11T15:53:30,605 adding 'lm_eval/tasks/bigbench/multiple_choice_template_a_yaml' 2026-05-11T15:53:30,607 adding 'lm_eval/tasks/bigbench/multiple_choice_template_b_yaml' 2026-05-11T15:53:30,608 adding 'lm_eval/tasks/bigbench/push_bigbench_dataset.py' 2026-05-11T15:53:30,613 adding 'lm_eval/tasks/bigbench/generate_until/abstract_narrative_understanding.yaml' 2026-05-11T15:53:30,614 adding 'lm_eval/tasks/bigbench/generate_until/anachronisms.yaml' 2026-05-11T15:53:30,615 adding 'lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml' 2026-05-11T15:53:30,616 adding 'lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml' 2026-05-11T15:53:30,617 adding 'lm_eval/tasks/bigbench/generate_until/arithmetic.yaml' 2026-05-11T15:53:30,618 adding 'lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml' 2026-05-11T15:53:30,619 adding 'lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml' 2026-05-11T15:53:30,621 adding 'lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml' 2026-05-11T15:53:30,622 adding 'lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml' 2026-05-11T15:53:30,623 adding 'lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml' 2026-05-11T15:53:30,624 adding 'lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml' 2026-05-11T15:53:30,626 adding 'lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml' 2026-05-11T15:53:30,627 adding 'lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml' 2026-05-11T15:53:30,628 adding 'lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml' 2026-05-11T15:53:30,629 adding 'lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml' 2026-05-11T15:53:30,630 adding 'lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml' 2026-05-11T15:53:30,632 adding 'lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml' 2026-05-11T15:53:30,633 adding 'lm_eval/tasks/bigbench/generate_until/code_line_description.yaml' 2026-05-11T15:53:30,634 adding 'lm_eval/tasks/bigbench/generate_until/codenames.yaml' 2026-05-11T15:53:30,635 adding 'lm_eval/tasks/bigbench/generate_until/color.yaml' 2026-05-11T15:53:30,636 adding 'lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml' 2026-05-11T15:53:30,638 adding 'lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml' 2026-05-11T15:53:30,639 adding 'lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml' 2026-05-11T15:53:30,640 adding 'lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml' 2026-05-11T15:53:30,641 adding 'lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml' 2026-05-11T15:53:30,642 adding 'lm_eval/tasks/bigbench/generate_until/crass_ai.yaml' 2026-05-11T15:53:30,644 adding 'lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml' 2026-05-11T15:53:30,645 adding 'lm_eval/tasks/bigbench/generate_until/cryptonite.yaml' 2026-05-11T15:53:30,646 adding 'lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml' 2026-05-11T15:53:30,647 adding 'lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml' 2026-05-11T15:53:30,649 adding 'lm_eval/tasks/bigbench/generate_until/date_understanding.yaml' 2026-05-11T15:53:30,650 adding 'lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml' 2026-05-11T15:53:30,651 adding 'lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml' 2026-05-11T15:53:30,653 adding 'lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml' 2026-05-11T15:53:30,654 adding 'lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml' 2026-05-11T15:53:30,655 adding 'lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml' 2026-05-11T15:53:30,657 adding 'lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml' 2026-05-11T15:53:30,658 adding 'lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml' 2026-05-11T15:53:30,659 adding 'lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml' 2026-05-11T15:53:30,660 adding 'lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml' 2026-05-11T15:53:30,661 adding 'lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml' 2026-05-11T15:53:30,662 adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml' 2026-05-11T15:53:30,663 adding 'lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml' 2026-05-11T15:53:30,664 adding 'lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml' 2026-05-11T15:53:30,665 adding 'lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml' 2026-05-11T15:53:30,666 adding 'lm_eval/tasks/bigbench/generate_until/fact_checker.yaml' 2026-05-11T15:53:30,668 adding 'lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml' 2026-05-11T15:53:30,669 adding 'lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml' 2026-05-11T15:53:30,670 adding 'lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml' 2026-05-11T15:53:30,671 adding 'lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml' 2026-05-11T15:53:30,672 adding 'lm_eval/tasks/bigbench/generate_until/gem.yaml' 2026-05-11T15:53:30,673 adding 'lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml' 2026-05-11T15:53:30,674 adding 'lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml' 2026-05-11T15:53:30,675 adding 'lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml' 2026-05-11T15:53:30,677 adding 'lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml' 2026-05-11T15:53:30,678 adding 'lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml' 2026-05-11T15:53:30,679 adding 'lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml' 2026-05-11T15:53:30,680 adding 'lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml' 2026-05-11T15:53:30,681 adding 'lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml' 2026-05-11T15:53:30,682 adding 'lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml' 2026-05-11T15:53:30,683 adding 'lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml' 2026-05-11T15:53:30,685 adding 'lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml' 2026-05-11T15:53:30,686 adding 'lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml' 2026-05-11T15:53:30,687 adding 'lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml' 2026-05-11T15:53:30,688 adding 'lm_eval/tasks/bigbench/generate_until/implicatures.yaml' 2026-05-11T15:53:30,689 adding 'lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml' 2026-05-11T15:53:30,690 adding 'lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml' 2026-05-11T15:53:30,692 adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml' 2026-05-11T15:53:30,693 adding 'lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml' 2026-05-11T15:53:30,694 adding 'lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml' 2026-05-11T15:53:30,695 adding 'lm_eval/tasks/bigbench/generate_until/irony_identification.yaml' 2026-05-11T15:53:30,696 adding 'lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml' 2026-05-11T15:53:30,698 adding 'lm_eval/tasks/bigbench/generate_until/kannada.yaml' 2026-05-11T15:53:30,699 adding 'lm_eval/tasks/bigbench/generate_until/key_value_maps.yaml' 2026-05-11T15:53:30,700 adding 'lm_eval/tasks/bigbench/generate_until/known_unknowns.yaml' 2026-05-11T15:53:30,701 adding 'lm_eval/tasks/bigbench/generate_until/language_games.yaml' 2026-05-11T15:53:30,702 adding 'lm_eval/tasks/bigbench/generate_until/language_identification.yaml' 2026-05-11T15:53:30,703 adding 'lm_eval/tasks/bigbench/generate_until/linguistic_mappings.yaml' 2026-05-11T15:53:30,705 adding 'lm_eval/tasks/bigbench/generate_until/linguistics_puzzles.yaml' 2026-05-11T15:53:30,706 adding 'lm_eval/tasks/bigbench/generate_until/list_functions.yaml' 2026-05-11T15:53:30,707 adding 'lm_eval/tasks/bigbench/generate_until/logic_grid_puzzle.yaml' 2026-05-11T15:53:30,708 adding 'lm_eval/tasks/bigbench/generate_until/logical_args.yaml' 2026-05-11T15:53:30,709 adding 'lm_eval/tasks/bigbench/generate_until/logical_deduction.yaml' 2026-05-11T15:53:30,710 adding 'lm_eval/tasks/bigbench/generate_until/logical_fallacy_detection.yaml' 2026-05-11T15:53:30,711 adding 'lm_eval/tasks/bigbench/generate_until/logical_sequence.yaml' 2026-05-11T15:53:30,712 adding 'lm_eval/tasks/bigbench/generate_until/mathematical_induction.yaml' 2026-05-11T15:53:30,714 adding 'lm_eval/tasks/bigbench/generate_until/matrixshapes.yaml' 2026-05-11T15:53:30,715 adding 'lm_eval/tasks/bigbench/generate_until/metaphor_boolean.yaml' 2026-05-11T15:53:30,716 adding 'lm_eval/tasks/bigbench/generate_until/metaphor_understanding.yaml' 2026-05-11T15:53:30,717 adding 'lm_eval/tasks/bigbench/generate_until/minute_mysteries_qa.yaml' 2026-05-11T15:53:30,718 adding 'lm_eval/tasks/bigbench/generate_until/misconceptions.yaml' 2026-05-11T15:53:30,719 adding 'lm_eval/tasks/bigbench/generate_until/misconceptions_russian.yaml' 2026-05-11T15:53:30,721 adding 'lm_eval/tasks/bigbench/generate_until/mnist_ascii.yaml' 2026-05-11T15:53:30,722 adding 'lm_eval/tasks/bigbench/generate_until/modified_arithmetic.yaml' 2026-05-11T15:53:30,723 adding 'lm_eval/tasks/bigbench/generate_until/moral_permissibility.yaml' 2026-05-11T15:53:30,724 adding 'lm_eval/tasks/bigbench/generate_until/movie_dialog_same_or_different.yaml' 2026-05-11T15:53:30,725 adding 'lm_eval/tasks/bigbench/generate_until/movie_recommendation.yaml' 2026-05-11T15:53:30,726 adding 'lm_eval/tasks/bigbench/generate_until/mult_data_wrangling.yaml' 2026-05-11T15:53:30,727 adding 'lm_eval/tasks/bigbench/generate_until/multiemo.yaml' 2026-05-11T15:53:30,729 adding 'lm_eval/tasks/bigbench/generate_until/natural_instructions.yaml' 2026-05-11T15:53:30,730 adding 'lm_eval/tasks/bigbench/generate_until/navigate.yaml' 2026-05-11T15:53:30,731 adding 'lm_eval/tasks/bigbench/generate_until/nonsense_words_grammar.yaml' 2026-05-11T15:53:30,732 adding 'lm_eval/tasks/bigbench/generate_until/novel_concepts.yaml' 2026-05-11T15:53:30,733 adding 'lm_eval/tasks/bigbench/generate_until/object_counting.yaml' 2026-05-11T15:53:30,735 adding 'lm_eval/tasks/bigbench/generate_until/odd_one_out.yaml' 2026-05-11T15:53:30,736 adding 'lm_eval/tasks/bigbench/generate_until/operators.yaml' 2026-05-11T15:53:30,737 adding 'lm_eval/tasks/bigbench/generate_until/paragraph_segmentation.yaml' 2026-05-11T15:53:30,738 adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_qa.yaml' 2026-05-11T15:53:30,739 adding 'lm_eval/tasks/bigbench/generate_until/parsinlu_reading_comprehension.yaml' 2026-05-11T15:53:30,741 adding 'lm_eval/tasks/bigbench/generate_until/penguins_in_a_table.yaml' 2026-05-11T15:53:30,742 adding 'lm_eval/tasks/bigbench/generate_until/periodic_elements.yaml' 2026-05-11T15:53:30,743 adding 'lm_eval/tasks/bigbench/generate_until/persian_idioms.yaml' 2026-05-11T15:53:30,744 adding 'lm_eval/tasks/bigbench/generate_until/phrase_relatedness.yaml' 2026-05-11T15:53:30,746 adding 'lm_eval/tasks/bigbench/generate_until/physical_intuition.yaml' 2026-05-11T15:53:30,747 adding 'lm_eval/tasks/bigbench/generate_until/physics.yaml' 2026-05-11T15:53:30,748 adding 'lm_eval/tasks/bigbench/generate_until/physics_questions.yaml' 2026-05-11T15:53:30,749 adding 'lm_eval/tasks/bigbench/generate_until/play_dialog_same_or_different.yaml' 2026-05-11T15:53:30,750 adding 'lm_eval/tasks/bigbench/generate_until/polish_sequence_labeling.yaml' 2026-05-11T15:53:30,752 adding 'lm_eval/tasks/bigbench/generate_until/presuppositions_as_nli.yaml' 2026-05-11T15:53:30,753 adding 'lm_eval/tasks/bigbench/generate_until/qa_wikidata.yaml' 2026-05-11T15:53:30,754 adding 'lm_eval/tasks/bigbench/generate_until/question_selection.yaml' 2026-05-11T15:53:30,756 adding 'lm_eval/tasks/bigbench/generate_until/real_or_fake_text.yaml' 2026-05-11T15:53:30,757 adding 'lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:30,758 adding 'lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml' 2026-05-11T15:53:30,759 adding 'lm_eval/tasks/bigbench/generate_until/rephrase.yaml' 2026-05-11T15:53:30,760 adding 'lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml' 2026-05-11T15:53:30,761 adding 'lm_eval/tasks/bigbench/generate_until/ruin_names.yaml' 2026-05-11T15:53:30,762 adding 'lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml' 2026-05-11T15:53:30,763 adding 'lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml' 2026-05-11T15:53:30,765 adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml' 2026-05-11T15:53:30,766 adding 'lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml' 2026-05-11T15:53:30,767 adding 'lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml' 2026-05-11T15:53:30,768 adding 'lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml' 2026-05-11T15:53:30,769 adding 'lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml' 2026-05-11T15:53:30,771 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml' 2026-05-11T15:53:30,772 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml' 2026-05-11T15:53:30,773 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml' 2026-05-11T15:53:30,774 adding 'lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml' 2026-05-11T15:53:30,775 adding 'lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml' 2026-05-11T15:53:30,776 adding 'lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml' 2026-05-11T15:53:30,777 adding 'lm_eval/tasks/bigbench/generate_until/snarks.yaml' 2026-05-11T15:53:30,779 adding 'lm_eval/tasks/bigbench/generate_until/social_iqa.yaml' 2026-05-11T15:53:30,780 adding 'lm_eval/tasks/bigbench/generate_until/social_support.yaml' 2026-05-11T15:53:30,781 adding 'lm_eval/tasks/bigbench/generate_until/sports_understanding.yaml' 2026-05-11T15:53:30,782 adding 'lm_eval/tasks/bigbench/generate_until/strange_stories.yaml' 2026-05-11T15:53:30,783 adding 'lm_eval/tasks/bigbench/generate_until/strategyqa.yaml' 2026-05-11T15:53:30,784 adding 'lm_eval/tasks/bigbench/generate_until/sufficient_information.yaml' 2026-05-11T15:53:30,786 adding 'lm_eval/tasks/bigbench/generate_until/suicide_risk.yaml' 2026-05-11T15:53:30,787 adding 'lm_eval/tasks/bigbench/generate_until/swahili_english_proverbs.yaml' 2026-05-11T15:53:30,788 adding 'lm_eval/tasks/bigbench/generate_until/swedish_to_german_proverbs.yaml' 2026-05-11T15:53:30,789 adding 'lm_eval/tasks/bigbench/generate_until/symbol_interpretation.yaml' 2026-05-11T15:53:30,790 adding 'lm_eval/tasks/bigbench/generate_until/temporal_sequences.yaml' 2026-05-11T15:53:30,791 adding 'lm_eval/tasks/bigbench/generate_until/tense.yaml' 2026-05-11T15:53:30,792 adding 'lm_eval/tasks/bigbench/generate_until/timedial.yaml' 2026-05-11T15:53:30,793 adding 'lm_eval/tasks/bigbench/generate_until/topical_chat.yaml' 2026-05-11T15:53:30,795 adding 'lm_eval/tasks/bigbench/generate_until/tracking_shuffled_objects.yaml' 2026-05-11T15:53:30,796 adding 'lm_eval/tasks/bigbench/generate_until/understanding_fables.yaml' 2026-05-11T15:53:30,797 adding 'lm_eval/tasks/bigbench/generate_until/undo_permutation.yaml' 2026-05-11T15:53:30,798 adding 'lm_eval/tasks/bigbench/generate_until/unit_conversion.yaml' 2026-05-11T15:53:30,799 adding 'lm_eval/tasks/bigbench/generate_until/unit_interpretation.yaml' 2026-05-11T15:53:30,800 adding 'lm_eval/tasks/bigbench/generate_until/unnatural_in_context_learning.yaml' 2026-05-11T15:53:30,802 adding 'lm_eval/tasks/bigbench/generate_until/vitaminc_fact_verification.yaml' 2026-05-11T15:53:30,803 adding 'lm_eval/tasks/bigbench/generate_until/what_is_the_tao.yaml' 2026-05-11T15:53:30,804 adding 'lm_eval/tasks/bigbench/generate_until/which_wiki_edit.yaml' 2026-05-11T15:53:30,805 adding 'lm_eval/tasks/bigbench/generate_until/winowhy.yaml' 2026-05-11T15:53:30,806 adding 'lm_eval/tasks/bigbench/generate_until/word_sorting.yaml' 2026-05-11T15:53:30,807 adding 'lm_eval/tasks/bigbench/generate_until/word_unscrambling.yaml' 2026-05-11T15:53:30,811 adding 'lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml' 2026-05-11T15:53:30,812 adding 'lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml' 2026-05-11T15:53:30,813 adding 'lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml' 2026-05-11T15:53:30,814 adding 'lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml' 2026-05-11T15:53:30,815 adding 'lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml' 2026-05-11T15:53:30,816 adding 'lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml' 2026-05-11T15:53:30,817 adding 'lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml' 2026-05-11T15:53:30,818 adding 'lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml' 2026-05-11T15:53:30,820 adding 'lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml' 2026-05-11T15:53:30,821 adding 'lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml' 2026-05-11T15:53:30,822 adding 'lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml' 2026-05-11T15:53:30,823 adding 'lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml' 2026-05-11T15:53:30,824 adding 'lm_eval/tasks/bigbench/multiple_choice/color.yaml' 2026-05-11T15:53:30,825 adding 'lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml' 2026-05-11T15:53:30,826 adding 'lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml' 2026-05-11T15:53:30,827 adding 'lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml' 2026-05-11T15:53:30,828 adding 'lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml' 2026-05-11T15:53:30,830 adding 'lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml' 2026-05-11T15:53:30,831 adding 'lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml' 2026-05-11T15:53:30,832 adding 'lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml' 2026-05-11T15:53:30,833 adding 'lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml' 2026-05-11T15:53:30,834 adding 'lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml' 2026-05-11T15:53:30,835 adding 'lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml' 2026-05-11T15:53:30,836 adding 'lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml' 2026-05-11T15:53:30,838 adding 'lm_eval/tasks/bigbench/multiple_choice/dyck_languages.yaml' 2026-05-11T15:53:30,839 adding 'lm_eval/tasks/bigbench/multiple_choice/elementary_math_qa.yaml' 2026-05-11T15:53:30,840 adding 'lm_eval/tasks/bigbench/multiple_choice/emoji_movie.yaml' 2026-05-11T15:53:30,841 adding 'lm_eval/tasks/bigbench/multiple_choice/emojis_emotion_prediction.yaml' 2026-05-11T15:53:30,842 adding 'lm_eval/tasks/bigbench/multiple_choice/empirical_judgments.yaml' 2026-05-11T15:53:30,843 adding 'lm_eval/tasks/bigbench/multiple_choice/english_proverbs.yaml' 2026-05-11T15:53:30,845 adding 'lm_eval/tasks/bigbench/multiple_choice/english_russian_proverbs.yaml' 2026-05-11T15:53:30,846 adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity.yaml' 2026-05-11T15:53:30,847 adding 'lm_eval/tasks/bigbench/multiple_choice/entailed_polarity_hindi.yaml' 2026-05-11T15:53:30,848 adding 'lm_eval/tasks/bigbench/multiple_choice/epistemic_reasoning.yaml' 2026-05-11T15:53:30,849 adding 'lm_eval/tasks/bigbench/multiple_choice/evaluating_information_essentiality.yaml' 2026-05-11T15:53:30,850 adding 'lm_eval/tasks/bigbench/multiple_choice/fact_checker.yaml' 2026-05-11T15:53:30,852 adding 'lm_eval/tasks/bigbench/multiple_choice/fantasy_reasoning.yaml' 2026-05-11T15:53:30,853 adding 'lm_eval/tasks/bigbench/multiple_choice/figure_of_speech_detection.yaml' 2026-05-11T15:53:30,854 adding 'lm_eval/tasks/bigbench/multiple_choice/formal_fallacies_syllogisms_negation.yaml' 2026-05-11T15:53:30,855 adding 'lm_eval/tasks/bigbench/multiple_choice/general_knowledge.yaml' 2026-05-11T15:53:30,856 adding 'lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml' 2026-05-11T15:53:30,857 adding 'lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml' 2026-05-11T15:53:30,858 adding 'lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml' 2026-05-11T15:53:30,859 adding 'lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml' 2026-05-11T15:53:30,860 adding 'lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml' 2026-05-11T15:53:30,861 adding 'lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml' 2026-05-11T15:53:30,863 adding 'lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml' 2026-05-11T15:53:30,864 adding 'lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml' 2026-05-11T15:53:30,865 adding 'lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml' 2026-05-11T15:53:30,866 adding 'lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml' 2026-05-11T15:53:30,867 adding 'lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml' 2026-05-11T15:53:30,868 adding 'lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml' 2026-05-11T15:53:30,870 adding 'lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml' 2026-05-11T15:53:30,871 adding 'lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml' 2026-05-11T15:53:30,872 adding 'lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml' 2026-05-11T15:53:30,873 adding 'lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml' 2026-05-11T15:53:30,874 adding 'lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml' 2026-05-11T15:53:30,875 adding 'lm_eval/tasks/bigbench/multiple_choice/kannada.yaml' 2026-05-11T15:53:30,876 adding 'lm_eval/tasks/bigbench/multiple_choice/key_value_maps.yaml' 2026-05-11T15:53:30,878 adding 'lm_eval/tasks/bigbench/multiple_choice/known_unknowns.yaml' 2026-05-11T15:53:30,879 adding 'lm_eval/tasks/bigbench/multiple_choice/language_identification.yaml' 2026-05-11T15:53:30,880 adding 'lm_eval/tasks/bigbench/multiple_choice/logic_grid_puzzle.yaml' 2026-05-11T15:53:30,881 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_args.yaml' 2026-05-11T15:53:30,883 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_deduction.yaml' 2026-05-11T15:53:30,884 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_fallacy_detection.yaml' 2026-05-11T15:53:30,885 adding 'lm_eval/tasks/bigbench/multiple_choice/logical_sequence.yaml' 2026-05-11T15:53:30,886 adding 'lm_eval/tasks/bigbench/multiple_choice/mathematical_induction.yaml' 2026-05-11T15:53:30,887 adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_boolean.yaml' 2026-05-11T15:53:30,889 adding 'lm_eval/tasks/bigbench/multiple_choice/metaphor_understanding.yaml' 2026-05-11T15:53:30,890 adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions.yaml' 2026-05-11T15:53:30,891 adding 'lm_eval/tasks/bigbench/multiple_choice/misconceptions_russian.yaml' 2026-05-11T15:53:30,892 adding 'lm_eval/tasks/bigbench/multiple_choice/mnist_ascii.yaml' 2026-05-11T15:53:30,893 adding 'lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml' 2026-05-11T15:53:30,894 adding 'lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml' 2026-05-11T15:53:30,896 adding 'lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml' 2026-05-11T15:53:30,897 adding 'lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml' 2026-05-11T15:53:30,898 adding 'lm_eval/tasks/bigbench/multiple_choice/navigate.yaml' 2026-05-11T15:53:30,899 adding 'lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml' 2026-05-11T15:53:30,901 adding 'lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml' 2026-05-11T15:53:30,902 adding 'lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml' 2026-05-11T15:53:30,903 adding 'lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml' 2026-05-11T15:53:30,904 adding 'lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml' 2026-05-11T15:53:30,905 adding 'lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml' 2026-05-11T15:53:30,906 adding 'lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml' 2026-05-11T15:53:30,907 adding 'lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml' 2026-05-11T15:53:30,908 adding 'lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml' 2026-05-11T15:53:30,909 adding 'lm_eval/tasks/bigbench/multiple_choice/physics.yaml' 2026-05-11T15:53:30,911 adding 'lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml' 2026-05-11T15:53:30,912 adding 'lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml' 2026-05-11T15:53:30,913 adding 'lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml' 2026-05-11T15:53:30,914 adding 'lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml' 2026-05-11T15:53:30,915 adding 'lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:30,916 adding 'lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml' 2026-05-11T15:53:30,917 adding 'lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml' 2026-05-11T15:53:30,918 adding 'lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml' 2026-05-11T15:53:30,919 adding 'lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml' 2026-05-11T15:53:30,921 adding 'lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml' 2026-05-11T15:53:30,922 adding 'lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml' 2026-05-11T15:53:30,923 adding 'lm_eval/tasks/bigbench/multiple_choice/snarks.yaml' 2026-05-11T15:53:30,924 adding 'lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml' 2026-05-11T15:53:30,925 adding 'lm_eval/tasks/bigbench/multiple_choice/social_support.yaml' 2026-05-11T15:53:30,926 adding 'lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml' 2026-05-11T15:53:30,927 adding 'lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml' 2026-05-11T15:53:30,929 adding 'lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml' 2026-05-11T15:53:30,930 adding 'lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml' 2026-05-11T15:53:30,931 adding 'lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml' 2026-05-11T15:53:30,932 adding 'lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml' 2026-05-11T15:53:30,933 adding 'lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml' 2026-05-11T15:53:30,935 adding 'lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml' 2026-05-11T15:53:30,936 adding 'lm_eval/tasks/bigbench/multiple_choice/timedial.yaml' 2026-05-11T15:53:30,937 adding 'lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml' 2026-05-11T15:53:30,938 adding 'lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml' 2026-05-11T15:53:30,939 adding 'lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml' 2026-05-11T15:53:30,941 adding 'lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml' 2026-05-11T15:53:30,942 adding 'lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml' 2026-05-11T15:53:30,943 adding 'lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml' 2026-05-11T15:53:30,944 adding 'lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml' 2026-05-11T15:53:30,945 adding 'lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml' 2026-05-11T15:53:30,946 adding 'lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml' 2026-05-11T15:53:30,950 adding 'lm_eval/tasks/blimp/README.md' 2026-05-11T15:53:30,951 adding 'lm_eval/tasks/blimp/_blimp.yaml' 2026-05-11T15:53:30,952 adding 'lm_eval/tasks/blimp/_template_yaml' 2026-05-11T15:53:30,953 adding 'lm_eval/tasks/blimp/adjunct_island.yaml' 2026-05-11T15:53:30,954 adding 'lm_eval/tasks/blimp/anaphor_gender_agreement.yaml' 2026-05-11T15:53:30,955 adding 'lm_eval/tasks/blimp/anaphor_number_agreement.yaml' 2026-05-11T15:53:30,956 adding 'lm_eval/tasks/blimp/animate_subject_passive.yaml' 2026-05-11T15:53:30,957 adding 'lm_eval/tasks/blimp/animate_subject_trans.yaml' 2026-05-11T15:53:30,958 adding 'lm_eval/tasks/blimp/causative.yaml' 2026-05-11T15:53:30,960 adding 'lm_eval/tasks/blimp/complex_NP_island.yaml' 2026-05-11T15:53:30,961 adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_complex_left_branch.yaml' 2026-05-11T15:53:30,962 adding 'lm_eval/tasks/blimp/coordinate_structure_constraint_object_extraction.yaml' 2026-05-11T15:53:30,963 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_1.yaml' 2026-05-11T15:53:30,964 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_2.yaml' 2026-05-11T15:53:30,965 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_1.yaml' 2026-05-11T15:53:30,966 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_irregular_2.yaml' 2026-05-11T15:53:30,968 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_2.yaml' 2026-05-11T15:53:30,969 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_1.yaml' 2026-05-11T15:53:30,970 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adj_irregular_2.yaml' 2026-05-11T15:53:30,971 adding 'lm_eval/tasks/blimp/determiner_noun_agreement_with_adjective_1.yaml' 2026-05-11T15:53:30,972 adding 'lm_eval/tasks/blimp/distractor_agreement_relational_noun.yaml' 2026-05-11T15:53:30,973 adding 'lm_eval/tasks/blimp/distractor_agreement_relative_clause.yaml' 2026-05-11T15:53:30,974 adding 'lm_eval/tasks/blimp/drop_argument.yaml' 2026-05-11T15:53:30,975 adding 'lm_eval/tasks/blimp/ellipsis_n_bar_1.yaml' 2026-05-11T15:53:30,977 adding 'lm_eval/tasks/blimp/ellipsis_n_bar_2.yaml' 2026-05-11T15:53:30,978 adding 'lm_eval/tasks/blimp/existential_there_object_raising.yaml' 2026-05-11T15:53:30,979 adding 'lm_eval/tasks/blimp/existential_there_quantifiers_1.yaml' 2026-05-11T15:53:30,980 adding 'lm_eval/tasks/blimp/existential_there_quantifiers_2.yaml' 2026-05-11T15:53:30,981 adding 'lm_eval/tasks/blimp/existential_there_subject_raising.yaml' 2026-05-11T15:53:30,982 adding 'lm_eval/tasks/blimp/expletive_it_object_raising.yaml' 2026-05-11T15:53:30,984 adding 'lm_eval/tasks/blimp/generate_configs.py' 2026-05-11T15:53:30,985 adding 'lm_eval/tasks/blimp/inchoative.yaml' 2026-05-11T15:53:30,986 adding 'lm_eval/tasks/blimp/intransitive.yaml' 2026-05-11T15:53:30,987 adding 'lm_eval/tasks/blimp/irregular_past_participle_adjectives.yaml' 2026-05-11T15:53:30,988 adding 'lm_eval/tasks/blimp/irregular_past_participle_verbs.yaml' 2026-05-11T15:53:30,990 adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_1.yaml' 2026-05-11T15:53:30,991 adding 'lm_eval/tasks/blimp/irregular_plural_subject_verb_agreement_2.yaml' 2026-05-11T15:53:30,992 adding 'lm_eval/tasks/blimp/left_branch_island_echo_question.yaml' 2026-05-11T15:53:30,993 adding 'lm_eval/tasks/blimp/left_branch_island_simple_question.yaml' 2026-05-11T15:53:30,994 adding 'lm_eval/tasks/blimp/matrix_question_npi_licensor_present.yaml' 2026-05-11T15:53:30,996 adding 'lm_eval/tasks/blimp/npi_present_1.yaml' 2026-05-11T15:53:30,997 adding 'lm_eval/tasks/blimp/npi_present_2.yaml' 2026-05-11T15:53:30,998 adding 'lm_eval/tasks/blimp/only_npi_licensor_present.yaml' 2026-05-11T15:53:30,999 adding 'lm_eval/tasks/blimp/only_npi_scope.yaml' 2026-05-11T15:53:31,000 adding 'lm_eval/tasks/blimp/passive_1.yaml' 2026-05-11T15:53:31,001 adding 'lm_eval/tasks/blimp/passive_2.yaml' 2026-05-11T15:53:31,003 adding 'lm_eval/tasks/blimp/principle_A_c_command.yaml' 2026-05-11T15:53:31,004 adding 'lm_eval/tasks/blimp/principle_A_case_1.yaml' 2026-05-11T15:53:31,005 adding 'lm_eval/tasks/blimp/principle_A_case_2.yaml' 2026-05-11T15:53:31,006 adding 'lm_eval/tasks/blimp/principle_A_domain_1.yaml' 2026-05-11T15:53:31,007 adding 'lm_eval/tasks/blimp/principle_A_domain_2.yaml' 2026-05-11T15:53:31,008 adding 'lm_eval/tasks/blimp/principle_A_domain_3.yaml' 2026-05-11T15:53:31,010 adding 'lm_eval/tasks/blimp/principle_A_reconstruction.yaml' 2026-05-11T15:53:31,011 adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_1.yaml' 2026-05-11T15:53:31,012 adding 'lm_eval/tasks/blimp/regular_plural_subject_verb_agreement_2.yaml' 2026-05-11T15:53:31,013 adding 'lm_eval/tasks/blimp/sentential_negation_npi_licensor_present.yaml' 2026-05-11T15:53:31,014 adding 'lm_eval/tasks/blimp/sentential_negation_npi_scope.yaml' 2026-05-11T15:53:31,015 adding 'lm_eval/tasks/blimp/sentential_subject_island.yaml' 2026-05-11T15:53:31,016 adding 'lm_eval/tasks/blimp/superlative_quantifiers_1.yaml' 2026-05-11T15:53:31,017 adding 'lm_eval/tasks/blimp/superlative_quantifiers_2.yaml' 2026-05-11T15:53:31,019 adding 'lm_eval/tasks/blimp/tough_vs_raising_1.yaml' 2026-05-11T15:53:31,020 adding 'lm_eval/tasks/blimp/tough_vs_raising_2.yaml' 2026-05-11T15:53:31,021 adding 'lm_eval/tasks/blimp/transitive.yaml' 2026-05-11T15:53:31,022 adding 'lm_eval/tasks/blimp/wh_island.yaml' 2026-05-11T15:53:31,023 adding 'lm_eval/tasks/blimp/wh_questions_object_gap.yaml' 2026-05-11T15:53:31,024 adding 'lm_eval/tasks/blimp/wh_questions_subject_gap.yaml' 2026-05-11T15:53:31,025 adding 'lm_eval/tasks/blimp/wh_questions_subject_gap_long_distance.yaml' 2026-05-11T15:53:31,026 adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap.yaml' 2026-05-11T15:53:31,027 adding 'lm_eval/tasks/blimp/wh_vs_that_no_gap_long_distance.yaml' 2026-05-11T15:53:31,029 adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap.yaml' 2026-05-11T15:53:31,030 adding 'lm_eval/tasks/blimp/wh_vs_that_with_gap_long_distance.yaml' 2026-05-11T15:53:31,034 adding 'lm_eval/tasks/blimp_nl/README.md' 2026-05-11T15:53:31,035 adding 'lm_eval/tasks/blimp_nl/_template_yaml' 2026-05-11T15:53:31,036 adding 'lm_eval/tasks/blimp_nl/adpositional_phrases__argument_r_extraction.yaml' 2026-05-11T15:53:31,037 adding 'lm_eval/tasks/blimp_nl/adpositional_phrases__argument_scrambling.yaml' 2026-05-11T15:53:31,038 adding 'lm_eval/tasks/blimp_nl/adverbial_modification__position_proform.yaml' 2026-05-11T15:53:31,039 adding 'lm_eval/tasks/blimp_nl/adverbial_modification__position_type.yaml' 2026-05-11T15:53:31,040 adding 'lm_eval/tasks/blimp_nl/anaphor_agreement__number.yaml' 2026-05-11T15:53:31,042 adding 'lm_eval/tasks/blimp_nl/anaphor_agreement__person.yaml' 2026-05-11T15:53:31,043 adding 'lm_eval/tasks/blimp_nl/argument_structure__argument_number_ditransitive.yaml' 2026-05-11T15:53:31,044 adding 'lm_eval/tasks/blimp_nl/argument_structure__argument_number_in_transitive.yaml' 2026-05-11T15:53:31,045 adding 'lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_1.yaml' 2026-05-11T15:53:31,046 adding 'lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_2.yaml' 2026-05-11T15:53:31,047 adding 'lm_eval/tasks/blimp_nl/argument_structure__ditransitive_nomdat_3.yaml' 2026-05-11T15:53:31,049 adding 'lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_1.yaml' 2026-05-11T15:53:31,050 adding 'lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_2.yaml' 2026-05-11T15:53:31,051 adding 'lm_eval/tasks/blimp_nl/argument_structure__intransitive_unaccusative_3.yaml' 2026-05-11T15:53:31,052 adding 'lm_eval/tasks/blimp_nl/auxiliaries__order_1.yaml' 2026-05-11T15:53:31,053 adding 'lm_eval/tasks/blimp_nl/auxiliaries__order_2.yaml' 2026-05-11T15:53:31,054 adding 'lm_eval/tasks/blimp_nl/auxiliaries__perfect.yaml' 2026-05-11T15:53:31,055 adding 'lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_1.yaml' 2026-05-11T15:53:31,057 adding 'lm_eval/tasks/blimp_nl/auxiliaries__semi_aspectual_2.yaml' 2026-05-11T15:53:31,058 adding 'lm_eval/tasks/blimp_nl/binding_principle_a__c_command.yaml' 2026-05-11T15:53:31,059 adding 'lm_eval/tasks/blimp_nl/binding_principle_a__monomorphemic.yaml' 2026-05-11T15:53:31,061 adding 'lm_eval/tasks/blimp_nl/blimp_nl_group.yaml' 2026-05-11T15:53:31,062 adding 'lm_eval/tasks/blimp_nl/complementive__ditransitive.yaml' 2026-05-11T15:53:31,063 adding 'lm_eval/tasks/blimp_nl/complementive__intransitive.yaml' 2026-05-11T15:53:31,064 adding 'lm_eval/tasks/blimp_nl/complementive__position_adverb.yaml' 2026-05-11T15:53:31,065 adding 'lm_eval/tasks/blimp_nl/complementive__position_verb.yaml' 2026-05-11T15:53:31,066 adding 'lm_eval/tasks/blimp_nl/complementive__transitive.yaml' 2026-05-11T15:53:31,067 adding 'lm_eval/tasks/blimp_nl/crossing_dependencies__cross_dependency.yaml' 2026-05-11T15:53:31,068 adding 'lm_eval/tasks/blimp_nl/determiners__geen_expletive.yaml' 2026-05-11T15:53:31,069 adding 'lm_eval/tasks/blimp_nl/determiners__geen_scrambling_1.yaml' 2026-05-11T15:53:31,070 adding 'lm_eval/tasks/blimp_nl/determiners__geen_scrambling_2.yaml' 2026-05-11T15:53:31,071 adding 'lm_eval/tasks/blimp_nl/determiners__negative_polarity.yaml' 2026-05-11T15:53:31,072 adding 'lm_eval/tasks/blimp_nl/extraposition__adjectival_adverbial.yaml' 2026-05-11T15:53:31,073 adding 'lm_eval/tasks/blimp_nl/extraposition__adjectival_supplementive.yaml' 2026-05-11T15:53:31,074 adding 'lm_eval/tasks/blimp_nl/extraposition__argument_nominal.yaml' 2026-05-11T15:53:31,075 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__complementizer.yaml' 2026-05-11T15:53:31,077 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__perception_dat.yaml' 2026-05-11T15:53:31,078 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml' 2026-05-11T15:53:31,079 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml' 2026-05-11T15:53:31,080 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml' 2026-05-11T15:53:31,081 adding 'lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml' 2026-05-11T15:53:31,082 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml' 2026-05-11T15:53:31,083 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml' 2026-05-11T15:53:31,084 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml' 2026-05-11T15:53:31,085 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml' 2026-05-11T15:53:31,087 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml' 2026-05-11T15:53:31,088 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml' 2026-05-11T15:53:31,089 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml' 2026-05-11T15:53:31,090 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml' 2026-05-11T15:53:31,091 adding 'lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml' 2026-05-11T15:53:31,092 adding 'lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml' 2026-05-11T15:53:31,093 adding 'lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml' 2026-05-11T15:53:31,094 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml' 2026-05-11T15:53:31,096 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml' 2026-05-11T15:53:31,097 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml' 2026-05-11T15:53:31,098 adding 'lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml' 2026-05-11T15:53:31,099 adding 'lm_eval/tasks/blimp_nl/passive__aci.yaml' 2026-05-11T15:53:31,100 adding 'lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml' 2026-05-11T15:53:31,101 adding 'lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml' 2026-05-11T15:53:31,102 adding 'lm_eval/tasks/blimp_nl/passive__impersonal.yaml' 2026-05-11T15:53:31,103 adding 'lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml' 2026-05-11T15:53:31,105 adding 'lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml' 2026-05-11T15:53:31,106 adding 'lm_eval/tasks/blimp_nl/r_words__adverbial.yaml' 2026-05-11T15:53:31,107 adding 'lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml' 2026-05-11T15:53:31,108 adding 'lm_eval/tasks/blimp_nl/relativization__island.yaml' 2026-05-11T15:53:31,109 adding 'lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml' 2026-05-11T15:53:31,110 adding 'lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml' 2026-05-11T15:53:31,111 adding 'lm_eval/tasks/blimp_nl/topicalization__island.yaml' 2026-05-11T15:53:31,112 adding 'lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml' 2026-05-11T15:53:31,113 adding 'lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml' 2026-05-11T15:53:31,114 adding 'lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml' 2026-05-11T15:53:31,115 adding 'lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml' 2026-05-11T15:53:31,116 adding 'lm_eval/tasks/blimp_nl/verb_second__order_main.yaml' 2026-05-11T15:53:31,117 adding 'lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml' 2026-05-11T15:53:31,118 adding 'lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml' 2026-05-11T15:53:31,119 adding 'lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml' 2026-05-11T15:53:31,120 adding 'lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml' 2026-05-11T15:53:31,121 adding 'lm_eval/tasks/blimp_nl/wh_movement__stranding_1.yaml' 2026-05-11T15:53:31,122 adding 'lm_eval/tasks/blimp_nl/wh_movement__stranding_2.yaml' 2026-05-11T15:53:31,123 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_1.yaml' 2026-05-11T15:53:31,125 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__bridge_verb_2.yaml' 2026-05-11T15:53:31,126 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_1.yaml' 2026-05-11T15:53:31,127 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__island_2.yaml' 2026-05-11T15:53:31,128 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__resumptive_prolepsis.yaml' 2026-05-11T15:53:31,129 adding 'lm_eval/tasks/blimp_nl/wh_movement_restrictions__superiority.yaml' 2026-05-11T15:53:31,131 adding 'lm_eval/tasks/c4/README.md' 2026-05-11T15:53:31,132 adding 'lm_eval/tasks/c4/c4.yaml' 2026-05-11T15:53:31,133 adding 'lm_eval/tasks/c4/preprocess_c4.py' 2026-05-11T15:53:31,135 adding 'lm_eval/tasks/cabbq/README.md' 2026-05-11T15:53:31,137 adding 'lm_eval/tasks/cabbq/_cabbq_common_yaml' 2026-05-11T15:53:31,138 adding 'lm_eval/tasks/cabbq/cabbq.yaml' 2026-05-11T15:53:31,139 adding 'lm_eval/tasks/cabbq/cabbq_age.yaml' 2026-05-11T15:53:31,140 adding 'lm_eval/tasks/cabbq/cabbq_disability_status.yaml' 2026-05-11T15:53:31,141 adding 'lm_eval/tasks/cabbq/cabbq_gender.yaml' 2026-05-11T15:53:31,142 adding 'lm_eval/tasks/cabbq/cabbq_lgbtqia.yaml' 2026-05-11T15:53:31,143 adding 'lm_eval/tasks/cabbq/cabbq_nationality.yaml' 2026-05-11T15:53:31,144 adding 'lm_eval/tasks/cabbq/cabbq_physical_appearance.yaml' 2026-05-11T15:53:31,146 adding 'lm_eval/tasks/cabbq/cabbq_race_ethnicity.yaml' 2026-05-11T15:53:31,147 adding 'lm_eval/tasks/cabbq/cabbq_religion.yaml' 2026-05-11T15:53:31,148 adding 'lm_eval/tasks/cabbq/cabbq_ses.yaml' 2026-05-11T15:53:31,149 adding 'lm_eval/tasks/cabbq/cabbq_spanish_region.yaml' 2026-05-11T15:53:31,151 adding 'lm_eval/tasks/cabbq/utils.py' 2026-05-11T15:53:31,153 adding 'lm_eval/tasks/careqa/README.md' 2026-05-11T15:53:31,154 adding 'lm_eval/tasks/careqa/careqa_en.yaml' 2026-05-11T15:53:31,155 adding 'lm_eval/tasks/careqa/careqa_es.yaml' 2026-05-11T15:53:31,157 adding 'lm_eval/tasks/careqa/careqa_open.yaml' 2026-05-11T15:53:31,158 adding 'lm_eval/tasks/careqa/careqa_open_perplexity.yaml' 2026-05-11T15:53:31,159 adding 'lm_eval/tasks/careqa/utils.py' 2026-05-11T15:53:31,160 adding 'lm_eval/tasks/careqa/utils_open.py' 2026-05-11T15:53:31,162 adding 'lm_eval/tasks/careqa/utils_perplexity.py' 2026-05-11T15:53:31,164 adding 'lm_eval/tasks/catalan_bench/README.md' 2026-05-11T15:53:31,165 adding 'lm_eval/tasks/catalan_bench/_arc_ca_common_yaml' 2026-05-11T15:53:31,167 adding 'lm_eval/tasks/catalan_bench/_cabreu_common_yaml' 2026-05-11T15:53:31,168 adding 'lm_eval/tasks/catalan_bench/arc_ca_challenge.yaml' 2026-05-11T15:53:31,169 adding 'lm_eval/tasks/catalan_bench/arc_ca_easy.yaml' 2026-05-11T15:53:31,170 adding 'lm_eval/tasks/catalan_bench/cabreu_abstractive.yaml' 2026-05-11T15:53:31,171 adding 'lm_eval/tasks/catalan_bench/cabreu_extractive.yaml' 2026-05-11T15:53:31,172 adding 'lm_eval/tasks/catalan_bench/cabreu_extreme.yaml' 2026-05-11T15:53:31,173 adding 'lm_eval/tasks/catalan_bench/catalan_bench.yaml' 2026-05-11T15:53:31,174 adding 'lm_eval/tasks/catalan_bench/catalanqa.yaml' 2026-05-11T15:53:31,175 adding 'lm_eval/tasks/catalan_bench/catcola.yaml' 2026-05-11T15:53:31,176 adding 'lm_eval/tasks/catalan_bench/cocoteros_va.yaml' 2026-05-11T15:53:31,177 adding 'lm_eval/tasks/catalan_bench/copa_ca.yaml' 2026-05-11T15:53:31,179 adding 'lm_eval/tasks/catalan_bench/coqcat.yaml' 2026-05-11T15:53:31,180 adding 'lm_eval/tasks/catalan_bench/mgsm_direct_ca.yaml' 2026-05-11T15:53:31,181 adding 'lm_eval/tasks/catalan_bench/openbookqa_ca.yaml' 2026-05-11T15:53:31,182 adding 'lm_eval/tasks/catalan_bench/parafraseja.yaml' 2026-05-11T15:53:31,184 adding 'lm_eval/tasks/catalan_bench/paws_ca.yaml' 2026-05-11T15:53:31,185 adding 'lm_eval/tasks/catalan_bench/piqa_ca.yaml' 2026-05-11T15:53:31,186 adding 'lm_eval/tasks/catalan_bench/siqa_ca.yaml' 2026-05-11T15:53:31,187 adding 'lm_eval/tasks/catalan_bench/teca.yaml' 2026-05-11T15:53:31,188 adding 'lm_eval/tasks/catalan_bench/utils.py' 2026-05-11T15:53:31,190 adding 'lm_eval/tasks/catalan_bench/wnli_ca.yaml' 2026-05-11T15:53:31,191 adding 'lm_eval/tasks/catalan_bench/xnli_ca.yaml' 2026-05-11T15:53:31,192 adding 'lm_eval/tasks/catalan_bench/xnli_va.yaml' 2026-05-11T15:53:31,193 adding 'lm_eval/tasks/catalan_bench/xquad_ca.yaml' 2026-05-11T15:53:31,194 adding 'lm_eval/tasks/catalan_bench/xstorycloze_ca.yaml' 2026-05-11T15:53:31,196 adding 'lm_eval/tasks/catalan_bench/flores_ca/_flores_common_yaml' 2026-05-11T15:53:31,198 adding 'lm_eval/tasks/catalan_bench/flores_ca/create_yamls_flores_ca.py' 2026-05-11T15:53:31,199 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-de.yaml' 2026-05-11T15:53:31,200 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-en.yaml' 2026-05-11T15:53:31,202 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-es.yaml' 2026-05-11T15:53:31,203 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-eu.yaml' 2026-05-11T15:53:31,204 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-fr.yaml' 2026-05-11T15:53:31,205 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-gl.yaml' 2026-05-11T15:53:31,206 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-it.yaml' 2026-05-11T15:53:31,207 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca-pt.yaml' 2026-05-11T15:53:31,209 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_ca.yaml' 2026-05-11T15:53:31,210 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_de-ca.yaml' 2026-05-11T15:53:31,211 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_en-ca.yaml' 2026-05-11T15:53:31,212 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_es-ca.yaml' 2026-05-11T15:53:31,213 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_eu-ca.yaml' 2026-05-11T15:53:31,215 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_fr-ca.yaml' 2026-05-11T15:53:31,216 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_gl-ca.yaml' 2026-05-11T15:53:31,217 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_it-ca.yaml' 2026-05-11T15:53:31,218 adding 'lm_eval/tasks/catalan_bench/flores_ca/flores_pt-ca.yaml' 2026-05-11T15:53:31,220 adding 'lm_eval/tasks/catalan_bench/phrases_va/_phrases_va_common' 2026-05-11T15:53:31,221 adding 'lm_eval/tasks/catalan_bench/phrases_va/phrases_ca-va.yaml' 2026-05-11T15:53:31,222 adding 'lm_eval/tasks/catalan_bench/phrases_va/phrases_va-ca.yaml' 2026-05-11T15:53:31,224 adding 'lm_eval/tasks/catalan_bench/truthfulqa_va/truthfulqa_va.yaml' 2026-05-11T15:53:31,226 adding 'lm_eval/tasks/catalan_bench/truthfulqa_va/utils.py' 2026-05-11T15:53:31,229 adding 'lm_eval/tasks/ceval/README.md' 2026-05-11T15:53:31,230 adding 'lm_eval/tasks/ceval/_ceval-valid.yaml' 2026-05-11T15:53:31,232 adding 'lm_eval/tasks/ceval/_default_ceval_yaml' 2026-05-11T15:53:31,233 adding 'lm_eval/tasks/ceval/_generate_configs.py' 2026-05-11T15:53:31,234 adding 'lm_eval/tasks/ceval/ceval-valid_accountant.yaml' 2026-05-11T15:53:31,235 adding 'lm_eval/tasks/ceval/ceval-valid_advanced_mathematics.yaml' 2026-05-11T15:53:31,237 adding 'lm_eval/tasks/ceval/ceval-valid_art_studies.yaml' 2026-05-11T15:53:31,238 adding 'lm_eval/tasks/ceval/ceval-valid_basic_medicine.yaml' 2026-05-11T15:53:31,239 adding 'lm_eval/tasks/ceval/ceval-valid_business_administration.yaml' 2026-05-11T15:53:31,240 adding 'lm_eval/tasks/ceval/ceval-valid_chinese_language_and_literature.yaml' 2026-05-11T15:53:31,241 adding 'lm_eval/tasks/ceval/ceval-valid_civil_servant.yaml' 2026-05-11T15:53:31,242 adding 'lm_eval/tasks/ceval/ceval-valid_clinical_medicine.yaml' 2026-05-11T15:53:31,243 adding 'lm_eval/tasks/ceval/ceval-valid_college_chemistry.yaml' 2026-05-11T15:53:31,245 adding 'lm_eval/tasks/ceval/ceval-valid_college_economics.yaml' 2026-05-11T15:53:31,246 adding 'lm_eval/tasks/ceval/ceval-valid_college_physics.yaml' 2026-05-11T15:53:31,247 adding 'lm_eval/tasks/ceval/ceval-valid_college_programming.yaml' 2026-05-11T15:53:31,248 adding 'lm_eval/tasks/ceval/ceval-valid_computer_architecture.yaml' 2026-05-11T15:53:31,249 adding 'lm_eval/tasks/ceval/ceval-valid_computer_network.yaml' 2026-05-11T15:53:31,250 adding 'lm_eval/tasks/ceval/ceval-valid_discrete_mathematics.yaml' 2026-05-11T15:53:31,251 adding 'lm_eval/tasks/ceval/ceval-valid_education_science.yaml' 2026-05-11T15:53:31,253 adding 'lm_eval/tasks/ceval/ceval-valid_electrical_engineer.yaml' 2026-05-11T15:53:31,254 adding 'lm_eval/tasks/ceval/ceval-valid_environmental_impact_assessment_engineer.yaml' 2026-05-11T15:53:31,255 adding 'lm_eval/tasks/ceval/ceval-valid_fire_engineer.yaml' 2026-05-11T15:53:31,256 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_biology.yaml' 2026-05-11T15:53:31,257 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chemistry.yaml' 2026-05-11T15:53:31,258 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_chinese.yaml' 2026-05-11T15:53:31,260 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_geography.yaml' 2026-05-11T15:53:31,261 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_history.yaml' 2026-05-11T15:53:31,262 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_mathematics.yaml' 2026-05-11T15:53:31,263 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_physics.yaml' 2026-05-11T15:53:31,265 adding 'lm_eval/tasks/ceval/ceval-valid_high_school_politics.yaml' 2026-05-11T15:53:31,266 adding 'lm_eval/tasks/ceval/ceval-valid_ideological_and_moral_cultivation.yaml' 2026-05-11T15:53:31,267 adding 'lm_eval/tasks/ceval/ceval-valid_law.yaml' 2026-05-11T15:53:31,268 adding 'lm_eval/tasks/ceval/ceval-valid_legal_professional.yaml' 2026-05-11T15:53:31,269 adding 'lm_eval/tasks/ceval/ceval-valid_logic.yaml' 2026-05-11T15:53:31,271 adding 'lm_eval/tasks/ceval/ceval-valid_mao_zedong_thought.yaml' 2026-05-11T15:53:31,272 adding 'lm_eval/tasks/ceval/ceval-valid_marxism.yaml' 2026-05-11T15:53:31,273 adding 'lm_eval/tasks/ceval/ceval-valid_metrology_engineer.yaml' 2026-05-11T15:53:31,274 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_biology.yaml' 2026-05-11T15:53:31,275 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_chemistry.yaml' 2026-05-11T15:53:31,277 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_geography.yaml' 2026-05-11T15:53:31,278 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_history.yaml' 2026-05-11T15:53:31,279 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_mathematics.yaml' 2026-05-11T15:53:31,280 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_physics.yaml' 2026-05-11T15:53:31,282 adding 'lm_eval/tasks/ceval/ceval-valid_middle_school_politics.yaml' 2026-05-11T15:53:31,283 adding 'lm_eval/tasks/ceval/ceval-valid_modern_chinese_history.yaml' 2026-05-11T15:53:31,284 adding 'lm_eval/tasks/ceval/ceval-valid_operating_system.yaml' 2026-05-11T15:53:31,285 adding 'lm_eval/tasks/ceval/ceval-valid_physician.yaml' 2026-05-11T15:53:31,286 adding 'lm_eval/tasks/ceval/ceval-valid_plant_protection.yaml' 2026-05-11T15:53:31,287 adding 'lm_eval/tasks/ceval/ceval-valid_probability_and_statistics.yaml' 2026-05-11T15:53:31,288 adding 'lm_eval/tasks/ceval/ceval-valid_professional_tour_guide.yaml' 2026-05-11T15:53:31,289 adding 'lm_eval/tasks/ceval/ceval-valid_sports_science.yaml' 2026-05-11T15:53:31,290 adding 'lm_eval/tasks/ceval/ceval-valid_tax_accountant.yaml' 2026-05-11T15:53:31,291 adding 'lm_eval/tasks/ceval/ceval-valid_teacher_qualification.yaml' 2026-05-11T15:53:31,292 adding 'lm_eval/tasks/ceval/ceval-valid_urban_and_rural_planner.yaml' 2026-05-11T15:53:31,293 adding 'lm_eval/tasks/ceval/ceval-valid_veterinary_medicine.yaml' 2026-05-11T15:53:31,295 adding 'lm_eval/tasks/chartqa/README.md' 2026-05-11T15:53:31,296 adding 'lm_eval/tasks/chartqa/chartqa.yaml' 2026-05-11T15:53:31,298 adding 'lm_eval/tasks/chartqa/chartqa_llama.yaml' 2026-05-11T15:53:31,299 adding 'lm_eval/tasks/chartqa/chartqa_llama_90.yaml' 2026-05-11T15:53:31,301 adding 'lm_eval/tasks/chartqa/utils.py' 2026-05-11T15:53:31,303 adding 'lm_eval/tasks/click/README.md' 2026-05-11T15:53:31,304 adding 'lm_eval/tasks/click/click.yaml' 2026-05-11T15:53:31,306 adding 'lm_eval/tasks/click/click_cul/_click_cul.yaml' 2026-05-11T15:53:31,307 adding 'lm_eval/tasks/click/click_cul/_default_click_cul_yaml' 2026-05-11T15:53:31,308 adding 'lm_eval/tasks/click/click_cul/click_cul_economy.yaml' 2026-05-11T15:53:31,309 adding 'lm_eval/tasks/click/click_cul/click_cul_geography.yaml' 2026-05-11T15:53:31,310 adding 'lm_eval/tasks/click/click_cul/click_cul_history.yaml' 2026-05-11T15:53:31,311 adding 'lm_eval/tasks/click/click_cul/click_cul_kpop.yaml' 2026-05-11T15:53:31,313 adding 'lm_eval/tasks/click/click_cul/click_cul_law.yaml' 2026-05-11T15:53:31,314 adding 'lm_eval/tasks/click/click_cul/click_cul_politics.yaml' 2026-05-11T15:53:31,315 adding 'lm_eval/tasks/click/click_cul/click_cul_society.yaml' 2026-05-11T15:53:31,316 adding 'lm_eval/tasks/click/click_cul/click_cul_tradition.yaml' 2026-05-11T15:53:31,317 adding 'lm_eval/tasks/click/click_cul/utils.py' 2026-05-11T15:53:31,319 adding 'lm_eval/tasks/click/click_lang/_click_lang.yaml' 2026-05-11T15:53:31,320 adding 'lm_eval/tasks/click/click_lang/_default_click_lang_yaml' 2026-05-11T15:53:31,322 adding 'lm_eval/tasks/click/click_lang/click_lang_function.yaml' 2026-05-11T15:53:31,323 adding 'lm_eval/tasks/click/click_lang/click_lang_grammar.yaml' 2026-05-11T15:53:31,324 adding 'lm_eval/tasks/click/click_lang/click_lang_text.yaml' 2026-05-11T15:53:31,325 adding 'lm_eval/tasks/click/click_lang/utils.py' 2026-05-11T15:53:31,329 adding 'lm_eval/tasks/cmmlu/README.md' 2026-05-11T15:53:31,330 adding 'lm_eval/tasks/cmmlu/_cmmlu.yaml' 2026-05-11T15:53:31,331 adding 'lm_eval/tasks/cmmlu/_default_template_yaml' 2026-05-11T15:53:31,333 adding 'lm_eval/tasks/cmmlu/_generate_configs.py' 2026-05-11T15:53:31,334 adding 'lm_eval/tasks/cmmlu/cmmlu_agronomy.yaml' 2026-05-11T15:53:31,335 adding 'lm_eval/tasks/cmmlu/cmmlu_anatomy.yaml' 2026-05-11T15:53:31,336 adding 'lm_eval/tasks/cmmlu/cmmlu_ancient_chinese.yaml' 2026-05-11T15:53:31,338 adding 'lm_eval/tasks/cmmlu/cmmlu_arts.yaml' 2026-05-11T15:53:31,339 adding 'lm_eval/tasks/cmmlu/cmmlu_astronomy.yaml' 2026-05-11T15:53:31,340 adding 'lm_eval/tasks/cmmlu/cmmlu_business_ethics.yaml' 2026-05-11T15:53:31,341 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_civil_service_exam.yaml' 2026-05-11T15:53:31,343 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_driving_rule.yaml' 2026-05-11T15:53:31,344 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_food_culture.yaml' 2026-05-11T15:53:31,345 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_foreign_policy.yaml' 2026-05-11T15:53:31,346 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_history.yaml' 2026-05-11T15:53:31,348 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_literature.yaml' 2026-05-11T15:53:31,349 adding 'lm_eval/tasks/cmmlu/cmmlu_chinese_teacher_qualification.yaml' 2026-05-11T15:53:31,350 adding 'lm_eval/tasks/cmmlu/cmmlu_clinical_knowledge.yaml' 2026-05-11T15:53:31,351 adding 'lm_eval/tasks/cmmlu/cmmlu_college_actuarial_science.yaml' 2026-05-11T15:53:31,352 adding 'lm_eval/tasks/cmmlu/cmmlu_college_education.yaml' 2026-05-11T15:53:31,353 adding 'lm_eval/tasks/cmmlu/cmmlu_college_engineering_hydrology.yaml' 2026-05-11T15:53:31,354 adding 'lm_eval/tasks/cmmlu/cmmlu_college_law.yaml' 2026-05-11T15:53:31,355 adding 'lm_eval/tasks/cmmlu/cmmlu_college_mathematics.yaml' 2026-05-11T15:53:31,356 adding 'lm_eval/tasks/cmmlu/cmmlu_college_medical_statistics.yaml' 2026-05-11T15:53:31,358 adding 'lm_eval/tasks/cmmlu/cmmlu_college_medicine.yaml' 2026-05-11T15:53:31,359 adding 'lm_eval/tasks/cmmlu/cmmlu_computer_science.yaml' 2026-05-11T15:53:31,360 adding 'lm_eval/tasks/cmmlu/cmmlu_computer_security.yaml' 2026-05-11T15:53:31,361 adding 'lm_eval/tasks/cmmlu/cmmlu_conceptual_physics.yaml' 2026-05-11T15:53:31,362 adding 'lm_eval/tasks/cmmlu/cmmlu_construction_project_management.yaml' 2026-05-11T15:53:31,364 adding 'lm_eval/tasks/cmmlu/cmmlu_economics.yaml' 2026-05-11T15:53:31,365 adding 'lm_eval/tasks/cmmlu/cmmlu_education.yaml' 2026-05-11T15:53:31,366 adding 'lm_eval/tasks/cmmlu/cmmlu_electrical_engineering.yaml' 2026-05-11T15:53:31,367 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_chinese.yaml' 2026-05-11T15:53:31,368 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_commonsense.yaml' 2026-05-11T15:53:31,370 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_information_and_technology.yaml' 2026-05-11T15:53:31,371 adding 'lm_eval/tasks/cmmlu/cmmlu_elementary_mathematics.yaml' 2026-05-11T15:53:31,372 adding 'lm_eval/tasks/cmmlu/cmmlu_ethnology.yaml' 2026-05-11T15:53:31,373 adding 'lm_eval/tasks/cmmlu/cmmlu_food_science.yaml' 2026-05-11T15:53:31,374 adding 'lm_eval/tasks/cmmlu/cmmlu_genetics.yaml' 2026-05-11T15:53:31,375 adding 'lm_eval/tasks/cmmlu/cmmlu_global_facts.yaml' 2026-05-11T15:53:31,377 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_biology.yaml' 2026-05-11T15:53:31,378 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_chemistry.yaml' 2026-05-11T15:53:31,379 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_geography.yaml' 2026-05-11T15:53:31,380 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_mathematics.yaml' 2026-05-11T15:53:31,381 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_physics.yaml' 2026-05-11T15:53:31,383 adding 'lm_eval/tasks/cmmlu/cmmlu_high_school_politics.yaml' 2026-05-11T15:53:31,384 adding 'lm_eval/tasks/cmmlu/cmmlu_human_sexuality.yaml' 2026-05-11T15:53:31,385 adding 'lm_eval/tasks/cmmlu/cmmlu_international_law.yaml' 2026-05-11T15:53:31,386 adding 'lm_eval/tasks/cmmlu/cmmlu_journalism.yaml' 2026-05-11T15:53:31,387 adding 'lm_eval/tasks/cmmlu/cmmlu_jurisprudence.yaml' 2026-05-11T15:53:31,388 adding 'lm_eval/tasks/cmmlu/cmmlu_legal_and_moral_basis.yaml' 2026-05-11T15:53:31,390 adding 'lm_eval/tasks/cmmlu/cmmlu_logical.yaml' 2026-05-11T15:53:31,391 adding 'lm_eval/tasks/cmmlu/cmmlu_machine_learning.yaml' 2026-05-11T15:53:31,392 adding 'lm_eval/tasks/cmmlu/cmmlu_management.yaml' 2026-05-11T15:53:31,393 adding 'lm_eval/tasks/cmmlu/cmmlu_marketing.yaml' 2026-05-11T15:53:31,394 adding 'lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml' 2026-05-11T15:53:31,395 adding 'lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml' 2026-05-11T15:53:31,397 adding 'lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml' 2026-05-11T15:53:31,398 adding 'lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml' 2026-05-11T15:53:31,399 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml' 2026-05-11T15:53:31,400 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml' 2026-05-11T15:53:31,401 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml' 2026-05-11T15:53:31,403 adding 'lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml' 2026-05-11T15:53:31,404 adding 'lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml' 2026-05-11T15:53:31,405 adding 'lm_eval/tasks/cmmlu/cmmlu_security_study.yaml' 2026-05-11T15:53:31,406 adding 'lm_eval/tasks/cmmlu/cmmlu_sociology.yaml' 2026-05-11T15:53:31,407 adding 'lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml' 2026-05-11T15:53:31,408 adding 'lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml' 2026-05-11T15:53:31,409 adding 'lm_eval/tasks/cmmlu/cmmlu_virology.yaml' 2026-05-11T15:53:31,410 adding 'lm_eval/tasks/cmmlu/cmmlu_world_history.yaml' 2026-05-11T15:53:31,411 adding 'lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml' 2026-05-11T15:53:31,413 adding 'lm_eval/tasks/cnn_dailymail/README.md' 2026-05-11T15:53:31,415 adding 'lm_eval/tasks/cnn_dailymail/cnn_dailymail.yaml' 2026-05-11T15:53:31,416 adding 'lm_eval/tasks/cnn_dailymail/utils.py' 2026-05-11T15:53:31,419 adding 'lm_eval/tasks/code_x_glue/code-text/README.md' 2026-05-11T15:53:31,420 adding 'lm_eval/tasks/code_x_glue/code-text/_codexglue.yaml' 2026-05-11T15:53:31,421 adding 'lm_eval/tasks/code_x_glue/code-text/_default_template_yaml' 2026-05-11T15:53:31,423 adding 'lm_eval/tasks/code_x_glue/code-text/bleu.py' 2026-05-11T15:53:31,424 adding 'lm_eval/tasks/code_x_glue/code-text/go.yaml' 2026-05-11T15:53:31,425 adding 'lm_eval/tasks/code_x_glue/code-text/java.yaml' 2026-05-11T15:53:31,426 adding 'lm_eval/tasks/code_x_glue/code-text/javascript.yaml' 2026-05-11T15:53:31,427 adding 'lm_eval/tasks/code_x_glue/code-text/php.yaml' 2026-05-11T15:53:31,428 adding 'lm_eval/tasks/code_x_glue/code-text/python.yaml' 2026-05-11T15:53:31,430 adding 'lm_eval/tasks/code_x_glue/code-text/ruby.yaml' 2026-05-11T15:53:31,431 adding 'lm_eval/tasks/code_x_glue/code-text/utils.py' 2026-05-11T15:53:31,433 adding 'lm_eval/tasks/common_voice/common_voice_en.yaml' 2026-05-11T15:53:31,434 adding 'lm_eval/tasks/common_voice/utils.py' 2026-05-11T15:53:31,436 adding 'lm_eval/tasks/commonsense_qa/README.md' 2026-05-11T15:53:31,437 adding 'lm_eval/tasks/commonsense_qa/default.yaml' 2026-05-11T15:53:31,439 adding 'lm_eval/tasks/copal_id/README.md' 2026-05-11T15:53:31,440 adding 'lm_eval/tasks/copal_id/colloquial.yaml' 2026-05-11T15:53:31,441 adding 'lm_eval/tasks/copal_id/standard.yaml' 2026-05-11T15:53:31,442 adding 'lm_eval/tasks/copal_id/utils.py' 2026-05-11T15:53:31,444 adding 'lm_eval/tasks/coqa/README.md' 2026-05-11T15:53:31,446 adding 'lm_eval/tasks/coqa/default.yaml' 2026-05-11T15:53:31,447 adding 'lm_eval/tasks/coqa/utils.py' 2026-05-11T15:53:31,450 adding 'lm_eval/tasks/crows_pairs/README.md' 2026-05-11T15:53:31,451 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english.yaml' 2026-05-11T15:53:31,452 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_age.yaml' 2026-05-11T15:53:31,453 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_autre.yaml' 2026-05-11T15:53:31,454 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_disability.yaml' 2026-05-11T15:53:31,455 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_gender.yaml' 2026-05-11T15:53:31,457 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_nationality.yaml' 2026-05-11T15:53:31,458 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_physical_appearance.yaml' 2026-05-11T15:53:31,459 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_race_color.yaml' 2026-05-11T15:53:31,460 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_religion.yaml' 2026-05-11T15:53:31,461 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_sexual_orientation.yaml' 2026-05-11T15:53:31,462 adding 'lm_eval/tasks/crows_pairs/crows_pairs_english_socioeconomic.yaml' 2026-05-11T15:53:31,464 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french.yaml' 2026-05-11T15:53:31,465 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_age.yaml' 2026-05-11T15:53:31,466 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_autre.yaml' 2026-05-11T15:53:31,467 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_disability.yaml' 2026-05-11T15:53:31,469 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_gender.yaml' 2026-05-11T15:53:31,470 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_nationality.yaml' 2026-05-11T15:53:31,471 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_physical_appearance.yaml' 2026-05-11T15:53:31,472 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_race_color.yaml' 2026-05-11T15:53:31,473 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_religion.yaml' 2026-05-11T15:53:31,474 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_sexual_orientation.yaml' 2026-05-11T15:53:31,475 adding 'lm_eval/tasks/crows_pairs/crows_pairs_french_socioeconomic.yaml' 2026-05-11T15:53:31,476 adding 'lm_eval/tasks/crows_pairs/utils.py' 2026-05-11T15:53:31,478 adding 'lm_eval/tasks/csatqa/_csatqa.yaml' 2026-05-11T15:53:31,479 adding 'lm_eval/tasks/csatqa/_default_csatqa_yaml' 2026-05-11T15:53:31,480 adding 'lm_eval/tasks/csatqa/_generate_configs.py' 2026-05-11T15:53:31,481 adding 'lm_eval/tasks/csatqa/csatqa_gr.yaml' 2026-05-11T15:53:31,483 adding 'lm_eval/tasks/csatqa/csatqa_li.yaml' 2026-05-11T15:53:31,484 adding 'lm_eval/tasks/csatqa/csatqa_rch.yaml' 2026-05-11T15:53:31,485 adding 'lm_eval/tasks/csatqa/csatqa_rcs.yaml' 2026-05-11T15:53:31,486 adding 'lm_eval/tasks/csatqa/csatqa_rcss.yaml' 2026-05-11T15:53:31,487 adding 'lm_eval/tasks/csatqa/csatqa_wr.yaml' 2026-05-11T15:53:31,488 adding 'lm_eval/tasks/csatqa/utils.py' 2026-05-11T15:53:31,490 adding 'lm_eval/tasks/darija_bench/README.md' 2026-05-11T15:53:31,492 adding 'lm_eval/tasks/darija_bench/darija_sentiment/README.md' 2026-05-11T15:53:31,493 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment.yaml' 2026-05-11T15:53:31,495 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_electrom.yaml' 2026-05-11T15:53:31,496 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_mac.yaml' 2026-05-11T15:53:31,497 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msac.yaml' 2026-05-11T15:53:31,498 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_msda.yaml' 2026-05-11T15:53:31,499 adding 'lm_eval/tasks/darija_bench/darija_sentiment/darija_sentiment_myc.yaml' 2026-05-11T15:53:31,500 adding 'lm_eval/tasks/darija_bench/darija_sentiment/default_darija_sentiment_template_yaml' 2026-05-11T15:53:31,502 adding 'lm_eval/tasks/darija_bench/darija_sentiment/utils.py' 2026-05-11T15:53:31,504 adding 'lm_eval/tasks/darija_bench/darija_summarization/README.md' 2026-05-11T15:53:31,505 adding 'lm_eval/tasks/darija_bench/darija_summarization/summarization.yaml' 2026-05-11T15:53:31,506 adding 'lm_eval/tasks/darija_bench/darija_summarization/summarization_common_yaml' 2026-05-11T15:53:31,507 adding 'lm_eval/tasks/darija_bench/darija_summarization/summarization_darija.yaml' 2026-05-11T15:53:31,509 adding 'lm_eval/tasks/darija_bench/darija_summarization/utils.py' 2026-05-11T15:53:31,511 adding 'lm_eval/tasks/darija_bench/darija_translation/README.md' 2026-05-11T15:53:31,512 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_common_yaml' 2026-05-11T15:53:31,514 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_all.yaml' 2026-05-11T15:53:31,515 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_darija.yaml' 2026-05-11T15:53:31,516 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_en.yaml' 2026-05-11T15:53:31,517 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_fr.yaml' 2026-05-11T15:53:31,518 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_dr_msa.yaml' 2026-05-11T15:53:31,519 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_en_dr.yaml' 2026-05-11T15:53:31,521 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_fr_dr.yaml' 2026-05-11T15:53:31,522 adding 'lm_eval/tasks/darija_bench/darija_translation/doda_translation_msa_dr.yaml' 2026-05-11T15:53:31,523 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_common_yaml' 2026-05-11T15:53:31,524 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_all.yaml' 2026-05-11T15:53:31,525 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_darija.yaml' 2026-05-11T15:53:31,526 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_en.yaml' 2026-05-11T15:53:31,527 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_fr.yaml' 2026-05-11T15:53:31,528 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_dr_msa.yaml' 2026-05-11T15:53:31,530 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_en_dr.yaml' 2026-05-11T15:53:31,531 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_fr_dr.yaml' 2026-05-11T15:53:31,532 adding 'lm_eval/tasks/darija_bench/darija_translation/flores_translation_msa_dr.yaml' 2026-05-11T15:53:31,533 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_common_yaml' 2026-05-11T15:53:31,534 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_all.yaml' 2026-05-11T15:53:31,535 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_darija.yaml' 2026-05-11T15:53:31,536 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_dr_msa.yaml' 2026-05-11T15:53:31,537 adding 'lm_eval/tasks/darija_bench/darija_translation/madar_translation_msa_dr.yaml' 2026-05-11T15:53:31,538 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_common_yaml' 2026-05-11T15:53:31,540 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_all.yaml' 2026-05-11T15:53:31,541 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_darija.yaml' 2026-05-11T15:53:31,542 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_dr_en.yaml' 2026-05-11T15:53:31,543 adding 'lm_eval/tasks/darija_bench/darija_translation/seed_translation_en_dr.yaml' 2026-05-11T15:53:31,544 adding 'lm_eval/tasks/darija_bench/darija_translation/translation_common_yaml' 2026-05-11T15:53:31,545 adding 'lm_eval/tasks/darija_bench/darija_translation/translation_darija.yaml' 2026-05-11T15:53:31,547 adding 'lm_eval/tasks/darija_bench/darija_translation/utils.py' 2026-05-11T15:53:31,549 adding 'lm_eval/tasks/darija_bench/darija_transliteration/README.md' 2026-05-11T15:53:31,550 adding 'lm_eval/tasks/darija_bench/darija_transliteration/translation_ar_dr.yaml' 2026-05-11T15:53:31,551 adding 'lm_eval/tasks/darija_bench/darija_transliteration/translation_dr_ar.yaml' 2026-05-11T15:53:31,552 adding 'lm_eval/tasks/darija_bench/darija_transliteration/transliteration_all.yaml' 2026-05-11T15:53:31,553 adding 'lm_eval/tasks/darija_bench/darija_transliteration/transliteration_common_yaml' 2026-05-11T15:53:31,554 adding 'lm_eval/tasks/darija_bench/darija_transliteration/transliteration_darija.yaml' 2026-05-11T15:53:31,556 adding 'lm_eval/tasks/darija_bench/darija_transliteration/utils.py' 2026-05-11T15:53:31,558 adding 'lm_eval/tasks/darijahellaswag/README.md' 2026-05-11T15:53:31,559 adding 'lm_eval/tasks/darijahellaswag/darijahellaswag.yaml' 2026-05-11T15:53:31,560 adding 'lm_eval/tasks/darijahellaswag/utils.py' 2026-05-11T15:53:31,563 adding 'lm_eval/tasks/darijammlu/README.md' 2026-05-11T15:53:31,564 adding 'lm_eval/tasks/darijammlu/_darijammlu.yaml' 2026-05-11T15:53:31,565 adding 'lm_eval/tasks/darijammlu/_darijammlu_ar_mmlu.yaml' 2026-05-11T15:53:31,566 adding 'lm_eval/tasks/darijammlu/_darijammlu_mmlu.yaml' 2026-05-11T15:53:31,567 adding 'lm_eval/tasks/darijammlu/_default_darijammlu_template_yaml' 2026-05-11T15:53:31,569 adding 'lm_eval/tasks/darijammlu/_generate_configs.py' 2026-05-11T15:53:31,570 adding 'lm_eval/tasks/darijammlu/darijammlu_accounting.yaml' 2026-05-11T15:53:31,571 adding 'lm_eval/tasks/darijammlu/darijammlu_arabic_language.yaml' 2026-05-11T15:53:31,572 adding 'lm_eval/tasks/darijammlu/darijammlu_arabic_language_(general).yaml' 2026-05-11T15:53:31,573 adding 'lm_eval/tasks/darijammlu/darijammlu_arabic_language_(grammar).yaml' 2026-05-11T15:53:31,575 adding 'lm_eval/tasks/darijammlu/darijammlu_biology.yaml' 2026-05-11T15:53:31,576 adding 'lm_eval/tasks/darijammlu/darijammlu_civics.yaml' 2026-05-11T15:53:31,577 adding 'lm_eval/tasks/darijammlu/darijammlu_computer_science.yaml' 2026-05-11T15:53:31,578 adding 'lm_eval/tasks/darijammlu/darijammlu_driving_test.yaml' 2026-05-11T15:53:31,579 adding 'lm_eval/tasks/darijammlu/darijammlu_economics.yaml' 2026-05-11T15:53:31,580 adding 'lm_eval/tasks/darijammlu/darijammlu_general_knowledge.yaml' 2026-05-11T15:53:31,581 adding 'lm_eval/tasks/darijammlu/darijammlu_geography.yaml' 2026-05-11T15:53:31,582 adding 'lm_eval/tasks/darijammlu/darijammlu_global_facts.yaml' 2026-05-11T15:53:31,583 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_european_history.yaml' 2026-05-11T15:53:31,584 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_geography.yaml' 2026-05-11T15:53:31,585 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:31,586 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_psychology.yaml' 2026-05-11T15:53:31,587 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_statistics.yaml' 2026-05-11T15:53:31,589 adding 'lm_eval/tasks/darijammlu/darijammlu_high_school_world_history.yaml' 2026-05-11T15:53:31,590 adding 'lm_eval/tasks/darijammlu/darijammlu_history.yaml' 2026-05-11T15:53:31,591 adding 'lm_eval/tasks/darijammlu/darijammlu_human_aging.yaml' 2026-05-11T15:53:31,592 adding 'lm_eval/tasks/darijammlu/darijammlu_international_law.yaml' 2026-05-11T15:53:31,593 adding 'lm_eval/tasks/darijammlu/darijammlu_islamic_studies.yaml' 2026-05-11T15:53:31,594 adding 'lm_eval/tasks/darijammlu/darijammlu_jurisprudence.yaml' 2026-05-11T15:53:31,595 adding 'lm_eval/tasks/darijammlu/darijammlu_law.yaml' 2026-05-11T15:53:31,596 adding 'lm_eval/tasks/darijammlu/darijammlu_logical_fallacies.yaml' 2026-05-11T15:53:31,597 adding 'lm_eval/tasks/darijammlu/darijammlu_management.yaml' 2026-05-11T15:53:31,599 adding 'lm_eval/tasks/darijammlu/darijammlu_management_ar.yaml' 2026-05-11T15:53:31,600 adding 'lm_eval/tasks/darijammlu/darijammlu_marketing.yaml' 2026-05-11T15:53:31,601 adding 'lm_eval/tasks/darijammlu/darijammlu_math.yaml' 2026-05-11T15:53:31,602 adding 'lm_eval/tasks/darijammlu/darijammlu_moral_disputes.yaml' 2026-05-11T15:53:31,603 adding 'lm_eval/tasks/darijammlu/darijammlu_moral_scenarios.yaml' 2026-05-11T15:53:31,604 adding 'lm_eval/tasks/darijammlu/darijammlu_natural_science.yaml' 2026-05-11T15:53:31,606 adding 'lm_eval/tasks/darijammlu/darijammlu_nutrition.yaml' 2026-05-11T15:53:31,607 adding 'lm_eval/tasks/darijammlu/darijammlu_philosophy.yaml' 2026-05-11T15:53:31,608 adding 'lm_eval/tasks/darijammlu/darijammlu_philosophy_ar.yaml' 2026-05-11T15:53:31,609 adding 'lm_eval/tasks/darijammlu/darijammlu_physics.yaml' 2026-05-11T15:53:31,610 adding 'lm_eval/tasks/darijammlu/darijammlu_political_science.yaml' 2026-05-11T15:53:31,611 adding 'lm_eval/tasks/darijammlu/darijammlu_professional_law.yaml' 2026-05-11T15:53:31,612 adding 'lm_eval/tasks/darijammlu/darijammlu_professional_psychology.yaml' 2026-05-11T15:53:31,614 adding 'lm_eval/tasks/darijammlu/darijammlu_public_relations.yaml' 2026-05-11T15:53:31,615 adding 'lm_eval/tasks/darijammlu/darijammlu_security_studies.yaml' 2026-05-11T15:53:31,616 adding 'lm_eval/tasks/darijammlu/darijammlu_social_science.yaml' 2026-05-11T15:53:31,617 adding 'lm_eval/tasks/darijammlu/darijammlu_sociology.yaml' 2026-05-11T15:53:31,618 adding 'lm_eval/tasks/darijammlu/darijammlu_world_religions.yaml' 2026-05-11T15:53:31,619 adding 'lm_eval/tasks/darijammlu/utils.py' 2026-05-11T15:53:31,621 adding 'lm_eval/tasks/discrim_eval/README.md' 2026-05-11T15:53:31,622 adding 'lm_eval/tasks/discrim_eval/discrim_eval_explicit.yaml' 2026-05-11T15:53:31,624 adding 'lm_eval/tasks/discrim_eval/discrim_eval_implicit.yaml' 2026-05-11T15:53:31,625 adding 'lm_eval/tasks/discrim_eval/utils.py' 2026-05-11T15:53:31,627 adding 'lm_eval/tasks/drop/README.md' 2026-05-11T15:53:31,628 adding 'lm_eval/tasks/drop/default.yaml' 2026-05-11T15:53:31,630 adding 'lm_eval/tasks/drop/utils.py' 2026-05-11T15:53:31,632 adding 'lm_eval/tasks/e2lmc/mmlu_early_training/README.md' 2026-05-11T15:53:31,633 adding 'lm_eval/tasks/e2lmc/mmlu_early_training/custom_metrics.py' 2026-05-11T15:53:31,634 adding 'lm_eval/tasks/e2lmc/mmlu_early_training/mmlu_early_training.yaml' 2026-05-11T15:53:31,636 adding 'lm_eval/tasks/e2lmc/noor/README.md' 2026-05-11T15:53:31,637 adding 'lm_eval/tasks/e2lmc/noor/_generate_configs.py' 2026-05-11T15:53:31,638 adding 'lm_eval/tasks/e2lmc/noor/_noor.yaml' 2026-05-11T15:53:31,639 adding 'lm_eval/tasks/e2lmc/noor/_noor_stem.yaml' 2026-05-11T15:53:31,640 adding 'lm_eval/tasks/e2lmc/noor/_noor_template' 2026-05-11T15:53:31,642 adding 'lm_eval/tasks/e2lmc/noor/noor_abstract_algebra.yaml' 2026-05-11T15:53:31,643 adding 'lm_eval/tasks/e2lmc/noor/noor_college_computer_science.yaml' 2026-05-11T15:53:31,644 adding 'lm_eval/tasks/e2lmc/noor/noor_college_mathematics.yaml' 2026-05-11T15:53:31,645 adding 'lm_eval/tasks/e2lmc/noor/noor_college_physics.yaml' 2026-05-11T15:53:31,646 adding 'lm_eval/tasks/e2lmc/noor/noor_conceptual_physics.yaml' 2026-05-11T15:53:31,647 adding 'lm_eval/tasks/e2lmc/noor/noor_electrical_engineering.yaml' 2026-05-11T15:53:31,648 adding 'lm_eval/tasks/e2lmc/noor/noor_elementary_mathematics.yaml' 2026-05-11T15:53:31,649 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_computer_science.yaml' 2026-05-11T15:53:31,651 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_mathematics.yaml' 2026-05-11T15:53:31,652 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_physics.yaml' 2026-05-11T15:53:31,653 adding 'lm_eval/tasks/e2lmc/noor/noor_high_school_statistics.yaml' 2026-05-11T15:53:31,654 adding 'lm_eval/tasks/e2lmc/noor/noor_machine_learning.yaml' 2026-05-11T15:53:31,656 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/README.md' 2026-05-11T15:53:31,657 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/_sciknoweval_mcqa.yaml' 2026-05-11T15:53:31,658 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/_var5shots_template_yaml' 2026-05-11T15:53:31,659 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Biology.yaml' 2026-05-11T15:53:31,661 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Chemistry.yaml' 2026-05-11T15:53:31,662 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Material.yaml' 2026-05-11T15:53:31,663 adding 'lm_eval/tasks/e2lmc/sciknoweval_mcqa/sciknoweval_Physics.yaml' 2026-05-11T15:53:31,665 adding 'lm_eval/tasks/egyhellaswag/README.md' 2026-05-11T15:53:31,666 adding 'lm_eval/tasks/egyhellaswag/egyhellaswag.yaml' 2026-05-11T15:53:31,667 adding 'lm_eval/tasks/egyhellaswag/utils.py' 2026-05-11T15:53:31,670 adding 'lm_eval/tasks/egymmlu/README.md' 2026-05-11T15:53:31,671 adding 'lm_eval/tasks/egymmlu/_default_egymmlu_template_yaml' 2026-05-11T15:53:31,672 adding 'lm_eval/tasks/egymmlu/_egymmlu.yaml' 2026-05-11T15:53:31,673 adding 'lm_eval/tasks/egymmlu/_egymmlu_ar_mmlu.yaml' 2026-05-11T15:53:31,674 adding 'lm_eval/tasks/egymmlu/_egymmlu_mmlu.yaml' 2026-05-11T15:53:31,676 adding 'lm_eval/tasks/egymmlu/_generate_configs.py' 2026-05-11T15:53:31,677 adding 'lm_eval/tasks/egymmlu/egymmlu_accounting.yaml' 2026-05-11T15:53:31,678 adding 'lm_eval/tasks/egymmlu/egymmlu_arabic_language.yaml' 2026-05-11T15:53:31,679 adding 'lm_eval/tasks/egymmlu/egymmlu_arabic_language_(general).yaml' 2026-05-11T15:53:31,680 adding 'lm_eval/tasks/egymmlu/egymmlu_arabic_language_(grammar).yaml' 2026-05-11T15:53:31,682 adding 'lm_eval/tasks/egymmlu/egymmlu_biology.yaml' 2026-05-11T15:53:31,683 adding 'lm_eval/tasks/egymmlu/egymmlu_civics.yaml' 2026-05-11T15:53:31,684 adding 'lm_eval/tasks/egymmlu/egymmlu_computer_science.yaml' 2026-05-11T15:53:31,685 adding 'lm_eval/tasks/egymmlu/egymmlu_driving_test.yaml' 2026-05-11T15:53:31,686 adding 'lm_eval/tasks/egymmlu/egymmlu_economics.yaml' 2026-05-11T15:53:31,688 adding 'lm_eval/tasks/egymmlu/egymmlu_general_knowledge.yaml' 2026-05-11T15:53:31,689 adding 'lm_eval/tasks/egymmlu/egymmlu_geography.yaml' 2026-05-11T15:53:31,690 adding 'lm_eval/tasks/egymmlu/egymmlu_global_facts.yaml' 2026-05-11T15:53:31,691 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_european_history.yaml' 2026-05-11T15:53:31,692 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_geography.yaml' 2026-05-11T15:53:31,693 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:31,694 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_psychology.yaml' 2026-05-11T15:53:31,695 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_statistics.yaml' 2026-05-11T15:53:31,696 adding 'lm_eval/tasks/egymmlu/egymmlu_high_school_world_history.yaml' 2026-05-11T15:53:31,697 adding 'lm_eval/tasks/egymmlu/egymmlu_history.yaml' 2026-05-11T15:53:31,698 adding 'lm_eval/tasks/egymmlu/egymmlu_human_aging.yaml' 2026-05-11T15:53:31,699 adding 'lm_eval/tasks/egymmlu/egymmlu_international_law.yaml' 2026-05-11T15:53:31,701 adding 'lm_eval/tasks/egymmlu/egymmlu_islamic_studies.yaml' 2026-05-11T15:53:31,702 adding 'lm_eval/tasks/egymmlu/egymmlu_jurisprudence.yaml' 2026-05-11T15:53:31,703 adding 'lm_eval/tasks/egymmlu/egymmlu_law.yaml' 2026-05-11T15:53:31,704 adding 'lm_eval/tasks/egymmlu/egymmlu_logical_fallacies.yaml' 2026-05-11T15:53:31,705 adding 'lm_eval/tasks/egymmlu/egymmlu_management.yaml' 2026-05-11T15:53:31,706 adding 'lm_eval/tasks/egymmlu/egymmlu_management_ar.yaml' 2026-05-11T15:53:31,707 adding 'lm_eval/tasks/egymmlu/egymmlu_marketing.yaml' 2026-05-11T15:53:31,708 adding 'lm_eval/tasks/egymmlu/egymmlu_math.yaml' 2026-05-11T15:53:31,709 adding 'lm_eval/tasks/egymmlu/egymmlu_moral_disputes.yaml' 2026-05-11T15:53:31,710 adding 'lm_eval/tasks/egymmlu/egymmlu_moral_scenarios.yaml' 2026-05-11T15:53:31,711 adding 'lm_eval/tasks/egymmlu/egymmlu_natural_science.yaml' 2026-05-11T15:53:31,713 adding 'lm_eval/tasks/egymmlu/egymmlu_nutrition.yaml' 2026-05-11T15:53:31,714 adding 'lm_eval/tasks/egymmlu/egymmlu_philosophy.yaml' 2026-05-11T15:53:31,715 adding 'lm_eval/tasks/egymmlu/egymmlu_philosophy_ar.yaml' 2026-05-11T15:53:31,716 adding 'lm_eval/tasks/egymmlu/egymmlu_physics.yaml' 2026-05-11T15:53:31,717 adding 'lm_eval/tasks/egymmlu/egymmlu_political_science.yaml' 2026-05-11T15:53:31,718 adding 'lm_eval/tasks/egymmlu/egymmlu_professional_law.yaml' 2026-05-11T15:53:31,719 adding 'lm_eval/tasks/egymmlu/egymmlu_professional_psychology.yaml' 2026-05-11T15:53:31,721 adding 'lm_eval/tasks/egymmlu/egymmlu_public_relations.yaml' 2026-05-11T15:53:31,722 adding 'lm_eval/tasks/egymmlu/egymmlu_security_studies.yaml' 2026-05-11T15:53:31,723 adding 'lm_eval/tasks/egymmlu/egymmlu_social_science.yaml' 2026-05-11T15:53:31,724 adding 'lm_eval/tasks/egymmlu/egymmlu_sociology.yaml' 2026-05-11T15:53:31,725 adding 'lm_eval/tasks/egymmlu/egymmlu_world_religions.yaml' 2026-05-11T15:53:31,726 adding 'lm_eval/tasks/egymmlu/utils.py' 2026-05-11T15:53:31,728 adding 'lm_eval/tasks/eq_bench/README.md' 2026-05-11T15:53:31,730 adding 'lm_eval/tasks/eq_bench/default.yaml' 2026-05-11T15:53:31,731 adding 'lm_eval/tasks/eq_bench/utils.py' 2026-05-11T15:53:31,733 adding 'lm_eval/tasks/eq_bench/multilingual/eqbench_ca.yaml' 2026-05-11T15:53:31,734 adding 'lm_eval/tasks/eq_bench/multilingual/eqbench_es.yaml' 2026-05-11T15:53:31,735 adding 'lm_eval/tasks/eq_bench/multilingual/utils.py' 2026-05-11T15:53:31,737 adding 'lm_eval/tasks/esbbq/README.md' 2026-05-11T15:53:31,738 adding 'lm_eval/tasks/esbbq/_esbbq_common_yaml' 2026-05-11T15:53:31,740 adding 'lm_eval/tasks/esbbq/esbbq.yaml' 2026-05-11T15:53:31,741 adding 'lm_eval/tasks/esbbq/esbbq_age.yaml' 2026-05-11T15:53:31,742 adding 'lm_eval/tasks/esbbq/esbbq_disability_status.yaml' 2026-05-11T15:53:31,743 adding 'lm_eval/tasks/esbbq/esbbq_gender.yaml' 2026-05-11T15:53:31,744 adding 'lm_eval/tasks/esbbq/esbbq_lgbtqia.yaml' 2026-05-11T15:53:31,745 adding 'lm_eval/tasks/esbbq/esbbq_nationality.yaml' 2026-05-11T15:53:31,746 adding 'lm_eval/tasks/esbbq/esbbq_physical_appearance.yaml' 2026-05-11T15:53:31,747 adding 'lm_eval/tasks/esbbq/esbbq_race_ethnicity.yaml' 2026-05-11T15:53:31,748 adding 'lm_eval/tasks/esbbq/esbbq_religion.yaml' 2026-05-11T15:53:31,749 adding 'lm_eval/tasks/esbbq/esbbq_ses.yaml' 2026-05-11T15:53:31,750 adding 'lm_eval/tasks/esbbq/esbbq_spanish_region.yaml' 2026-05-11T15:53:31,752 adding 'lm_eval/tasks/esbbq/utils.py' 2026-05-11T15:53:31,755 adding 'lm_eval/tasks/eus_exams/README.md' 2026-05-11T15:53:31,756 adding 'lm_eval/tasks/eus_exams/configs.py' 2026-05-11T15:53:31,757 adding 'lm_eval/tasks/eus_exams/eus_exams' 2026-05-11T15:53:31,758 adding 'lm_eval/tasks/eus_exams/eus_exams_es' 2026-05-11T15:53:31,759 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejadministrativo.yaml' 2026-05-11T15:53:31,760 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejauxiliar.yaml' 2026-05-11T15:53:31,762 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejsubalterno.yaml' 2026-05-11T15:53:31,763 adding 'lm_eval/tasks/eus_exams/eus_exams_es_ejtecnico.yaml' 2026-05-11T15:53:31,764 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeayuntamientovitoria.yaml' 2026-05-11T15:53:31,765 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opebilbao.yaml' 2026-05-11T15:53:31,766 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuadmin.yaml' 2026-05-11T15:53:31,767 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuaux.yaml' 2026-05-11T15:53:31,768 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehubiblio.yaml' 2026-05-11T15:53:31,769 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuderecho.yaml' 2026-05-11T15:53:31,770 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehueconomicas.yaml' 2026-05-11T15:53:31,771 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehuempresariales.yaml' 2026-05-11T15:53:31,773 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehusubalterno.yaml' 2026-05-11T15:53:31,774 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnico.yaml' 2026-05-11T15:53:31,775 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeehutecnicob.yaml' 2026-05-11T15:53:31,776 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiadmin.yaml' 2026-05-11T15:53:31,777 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiaux.yaml' 2026-05-11T15:53:31,778 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakiauxenf.yaml' 2026-05-11T15:53:31,779 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakicelador.yaml' 2026-05-11T15:53:31,780 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakienf.yaml' 2026-05-11T15:53:31,782 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakijuridico.yaml' 2026-05-11T15:53:31,783 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakioperario.yaml' 2026-05-11T15:53:31,784 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakitecnico.yaml' 2026-05-11T15:53:31,785 adding 'lm_eval/tasks/eus_exams/eus_exams_es_opeosakivarios.yaml' 2026-05-11T15:53:31,786 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza1c.yaml' 2026-05-11T15:53:31,787 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza2c.yaml' 2026-05-11T15:53:31,788 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza3c.yaml' 2026-05-11T15:53:31,790 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza4c.yaml' 2026-05-11T15:53:31,791 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza5c.yaml' 2026-05-11T15:53:31,792 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza6c.yaml' 2026-05-11T15:53:31,793 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza7c.yaml' 2026-05-11T15:53:31,794 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza8c.yaml' 2026-05-11T15:53:31,795 adding 'lm_eval/tasks/eus_exams/eus_exams_es_osakidetza9c.yaml' 2026-05-11T15:53:31,796 adding 'lm_eval/tasks/eus_exams/eus_exams_eu' 2026-05-11T15:53:31,798 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejadministrari.yaml' 2026-05-11T15:53:31,799 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntza.yaml' 2026-05-11T15:53:31,800 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejlaguntzaile.yaml' 2026-05-11T15:53:31,801 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_ejteknikari.yaml' 2026-05-11T15:53:31,802 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opebilbaoeu.yaml' 2026-05-11T15:53:31,803 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuadmineu.yaml' 2026-05-11T15:53:31,804 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuauxeu.yaml' 2026-05-11T15:53:31,805 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehubiblioeu.yaml' 2026-05-11T15:53:31,806 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuderechoeu.yaml' 2026-05-11T15:53:31,807 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehueconomicaseu.yaml' 2026-05-11T15:53:31,808 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuempresarialeseu.yaml' 2026-05-11T15:53:31,809 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehusubalternoeu.yaml' 2026-05-11T15:53:31,810 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehutecnicoeu.yaml' 2026-05-11T15:53:31,811 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeehuteknikarib.yaml' 2026-05-11T15:53:31,812 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opegasteizkoudala.yaml' 2026-05-11T15:53:31,814 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiadmineu.yaml' 2026-05-11T15:53:31,815 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxenfeu.yaml' 2026-05-11T15:53:31,816 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiauxeu.yaml' 2026-05-11T15:53:31,817 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakiceladoreu.yaml' 2026-05-11T15:53:31,818 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakienfeu.yaml' 2026-05-11T15:53:31,819 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakioperarioeu.yaml' 2026-05-11T15:53:31,820 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakitecnicoeu.yaml' 2026-05-11T15:53:31,821 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_opeosakivarioseu.yaml' 2026-05-11T15:53:31,822 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza1e.yaml' 2026-05-11T15:53:31,824 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza2e.yaml' 2026-05-11T15:53:31,825 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza3e.yaml' 2026-05-11T15:53:31,826 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza5e.yaml' 2026-05-11T15:53:31,827 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza6e.yaml' 2026-05-11T15:53:31,828 adding 'lm_eval/tasks/eus_exams/eus_exams_eu_osakidetza7e.yaml' 2026-05-11T15:53:31,829 adding 'lm_eval/tasks/eus_exams/utils.py' 2026-05-11T15:53:31,831 adding 'lm_eval/tasks/eus_proficiency/README.md' 2026-05-11T15:53:31,832 adding 'lm_eval/tasks/eus_proficiency/eus_proficiency.yaml' 2026-05-11T15:53:31,834 adding 'lm_eval/tasks/eus_reading/README.md' 2026-05-11T15:53:31,835 adding 'lm_eval/tasks/eus_reading/eus_reading.yaml' 2026-05-11T15:53:31,836 adding 'lm_eval/tasks/eus_reading/utils.py' 2026-05-11T15:53:31,838 adding 'lm_eval/tasks/eus_trivia/README.md' 2026-05-11T15:53:31,839 adding 'lm_eval/tasks/eus_trivia/eus_trivia.yaml' 2026-05-11T15:53:31,841 adding 'lm_eval/tasks/eus_trivia/utils.py' 2026-05-11T15:53:31,844 adding 'lm_eval/tasks/evalita_llm/README.md' 2026-05-11T15:53:31,845 adding 'lm_eval/tasks/evalita_llm/_at_template_yaml' 2026-05-11T15:53:31,846 adding 'lm_eval/tasks/evalita_llm/_evalita-mp.yaml' 2026-05-11T15:53:31,847 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p1.yaml' 2026-05-11T15:53:31,849 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p2.yaml' 2026-05-11T15:53:31,850 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p3.yaml' 2026-05-11T15:53:31,851 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p4.yaml' 2026-05-11T15:53:31,853 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p5.yaml' 2026-05-11T15:53:31,854 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_task_p6.yaml' 2026-05-11T15:53:31,855 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_at_tasks.yaml' 2026-05-11T15:53:31,856 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p1.yaml' 2026-05-11T15:53:31,857 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p2.yaml' 2026-05-11T15:53:31,859 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p3.yaml' 2026-05-11T15:53:31,860 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p4.yaml' 2026-05-11T15:53:31,861 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p5.yaml' 2026-05-11T15:53:31,862 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_p6.yaml' 2026-05-11T15:53:31,863 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_faq_tasks.yaml' 2026-05-11T15:53:31,864 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_gen.yaml' 2026-05-11T15:53:31,865 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p1.yaml' 2026-05-11T15:53:31,866 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p2.yaml' 2026-05-11T15:53:31,867 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p3.yaml' 2026-05-11T15:53:31,868 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p4.yaml' 2026-05-11T15:53:31,870 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p5.yaml' 2026-05-11T15:53:31,871 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_p6.yaml' 2026-05-11T15:53:31,872 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_hs_task.yaml' 2026-05-11T15:53:31,873 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ls_p1.yaml' 2026-05-11T15:53:31,874 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ls_p2.yaml' 2026-05-11T15:53:31,875 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ls_task.yaml' 2026-05-11T15:53:31,877 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_mc.yaml' 2026-05-11T15:53:31,878 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group.yaml' 2026-05-11T15:53:31,879 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p1.yaml' 2026-05-11T15:53:31,880 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-adg_group_p2.yaml' 2026-05-11T15:53:31,881 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group.yaml' 2026-05-11T15:53:31,882 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p1.yaml' 2026-05-11T15:53:31,884 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-fic_group_p2.yaml' 2026-05-11T15:53:31,885 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group.yaml' 2026-05-11T15:53:31,886 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p1.yaml' 2026-05-11T15:53:31,887 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner-wn_group_p2.yaml' 2026-05-11T15:53:31,888 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg' 2026-05-11T15:53:31,889 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p1.yaml' 2026-05-11T15:53:31,891 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_adg_p2.yaml' 2026-05-11T15:53:31,892 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic' 2026-05-11T15:53:31,893 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p1.yaml' 2026-05-11T15:53:31,894 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_fic_p2.yaml' 2026-05-11T15:53:31,895 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_group.yaml' 2026-05-11T15:53:31,896 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn' 2026-05-11T15:53:31,897 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p1.yaml' 2026-05-11T15:53:31,899 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_ner_wn_p2.yaml' 2026-05-11T15:53:31,900 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_re_p1.yaml' 2026-05-11T15:53:31,901 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_re_p2.yaml' 2026-05-11T15:53:31,902 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_re_task.yaml' 2026-05-11T15:53:31,903 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p1.yaml' 2026-05-11T15:53:31,905 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p2.yaml' 2026-05-11T15:53:31,906 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p3.yaml' 2026-05-11T15:53:31,907 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p4.yaml' 2026-05-11T15:53:31,908 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p5.yaml' 2026-05-11T15:53:31,909 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_p6.yaml' 2026-05-11T15:53:31,911 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sa_tasks.yaml' 2026-05-11T15:53:31,912 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p1.yaml' 2026-05-11T15:53:31,913 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_p2.yaml' 2026-05-11T15:53:31,914 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp-small_task.yaml' 2026-05-11T15:53:31,916 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p1.yaml' 2026-05-11T15:53:31,917 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_p2.yaml' 2026-05-11T15:53:31,918 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_sum_fp_task.yaml' 2026-05-11T15:53:31,919 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p1.yaml' 2026-05-11T15:53:31,920 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p2.yaml' 2026-05-11T15:53:31,921 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p3.yaml' 2026-05-11T15:53:31,922 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p4.yaml' 2026-05-11T15:53:31,923 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p5.yaml' 2026-05-11T15:53:31,924 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_p6.yaml' 2026-05-11T15:53:31,925 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_te_tasks.yaml' 2026-05-11T15:53:31,926 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p1.yaml' 2026-05-11T15:53:31,928 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p2.yaml' 2026-05-11T15:53:31,929 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p3.yaml' 2026-05-11T15:53:31,930 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p4.yaml' 2026-05-11T15:53:31,931 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p5.yaml' 2026-05-11T15:53:31,932 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_p6.yaml' 2026-05-11T15:53:31,933 adding 'lm_eval/tasks/evalita_llm/_evalita-mp_wic_tasks.yaml' 2026-05-11T15:53:31,934 adding 'lm_eval/tasks/evalita_llm/_faq_template_yaml' 2026-05-11T15:53:31,936 adding 'lm_eval/tasks/evalita_llm/_hs_template_yaml' 2026-05-11T15:53:31,937 adding 'lm_eval/tasks/evalita_llm/_ls_template_yaml' 2026-05-11T15:53:31,938 adding 'lm_eval/tasks/evalita_llm/_ner_template_yaml' 2026-05-11T15:53:31,939 adding 'lm_eval/tasks/evalita_llm/_re_template_yaml' 2026-05-11T15:53:31,940 adding 'lm_eval/tasks/evalita_llm/_sa_template_v2_yaml' 2026-05-11T15:53:31,941 adding 'lm_eval/tasks/evalita_llm/_sa_template_yaml' 2026-05-11T15:53:31,943 adding 'lm_eval/tasks/evalita_llm/_sum_template_fp-small_yaml' 2026-05-11T15:53:31,944 adding 'lm_eval/tasks/evalita_llm/_sum_template_fp_yaml' 2026-05-11T15:53:31,945 adding 'lm_eval/tasks/evalita_llm/_sum_template_yaml' 2026-05-11T15:53:31,946 adding 'lm_eval/tasks/evalita_llm/_te_template_yaml' 2026-05-11T15:53:31,947 adding 'lm_eval/tasks/evalita_llm/_wic_template_yaml' 2026-05-11T15:53:31,949 adding 'lm_eval/tasks/evalita_llm/metrics.py' 2026-05-11T15:53:31,950 adding 'lm_eval/tasks/evalita_llm/sum_utils.py' 2026-05-11T15:53:31,952 adding 'lm_eval/tasks/evalita_llm/utils.py' 2026-05-11T15:53:31,955 adding 'lm_eval/tasks/fda/README.md' 2026-05-11T15:53:31,956 adding 'lm_eval/tasks/fda/fda.yaml' 2026-05-11T15:53:31,957 adding 'lm_eval/tasks/fda/task.py' 2026-05-11T15:53:31,959 adding 'lm_eval/tasks/fld/README.md' 2026-05-11T15:53:31,961 adding 'lm_eval/tasks/fld/fld_default.yaml' 2026-05-11T15:53:31,962 adding 'lm_eval/tasks/fld/fld_logical_formula_default.yaml' 2026-05-11T15:53:31,963 adding 'lm_eval/tasks/fld/fld_logical_formula_star.yaml' 2026-05-11T15:53:31,965 adding 'lm_eval/tasks/fld/fld_star.yaml' 2026-05-11T15:53:31,967 adding 'lm_eval/tasks/french_bench/README.md' 2026-05-11T15:53:31,968 adding 'lm_eval/tasks/french_bench/_default_template_yaml' 2026-05-11T15:53:31,969 adding 'lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml' 2026-05-11T15:53:31,971 adding 'lm_eval/tasks/french_bench/french_bench_boolqa.yaml' 2026-05-11T15:53:31,972 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2.yaml' 2026-05-11T15:53:31,973 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml' 2026-05-11T15:53:31,974 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml' 2026-05-11T15:53:31,976 adding 'lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml' 2026-05-11T15:53:31,977 adding 'lm_eval/tasks/french_bench/french_bench_grammar.yaml' 2026-05-11T15:53:31,978 adding 'lm_eval/tasks/french_bench/french_bench_hellaswag.yaml' 2026-05-11T15:53:31,980 adding 'lm_eval/tasks/french_bench/french_bench_multifquad.yaml' 2026-05-11T15:53:31,981 adding 'lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml' 2026-05-11T15:53:31,982 adding 'lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml' 2026-05-11T15:53:31,983 adding 'lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml' 2026-05-11T15:53:31,984 adding 'lm_eval/tasks/french_bench/french_bench_reading_comp.yaml' 2026-05-11T15:53:31,985 adding 'lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml' 2026-05-11T15:53:31,987 adding 'lm_eval/tasks/french_bench/french_bench_trivia.yaml' 2026-05-11T15:53:31,988 adding 'lm_eval/tasks/french_bench/french_bench_vocab.yaml' 2026-05-11T15:53:31,989 adding 'lm_eval/tasks/french_bench/french_bench_wikitext_fr.yaml' 2026-05-11T15:53:31,991 adding 'lm_eval/tasks/french_bench/french_bench_xnli.yaml' 2026-05-11T15:53:31,992 adding 'lm_eval/tasks/french_bench/preprocess_wikitext.py' 2026-05-11T15:53:31,993 adding 'lm_eval/tasks/french_bench/utils.py' 2026-05-11T15:53:31,995 adding 'lm_eval/tasks/galician_bench/README.md' 2026-05-11T15:53:31,996 adding 'lm_eval/tasks/galician_bench/belebele_glg_Latn.yaml' 2026-05-11T15:53:31,998 adding 'lm_eval/tasks/galician_bench/galcola.yaml' 2026-05-11T15:53:31,999 adding 'lm_eval/tasks/galician_bench/galician_bench.yaml' 2026-05-11T15:53:32,000 adding 'lm_eval/tasks/galician_bench/mgsm_direct_gl.yaml' 2026-05-11T15:53:32,001 adding 'lm_eval/tasks/galician_bench/openbookqa_gl.yaml' 2026-05-11T15:53:32,002 adding 'lm_eval/tasks/galician_bench/parafrases_gl.yaml' 2026-05-11T15:53:32,003 adding 'lm_eval/tasks/galician_bench/paws_gl.yaml' 2026-05-11T15:53:32,004 adding 'lm_eval/tasks/galician_bench/summarization_gl.yaml' 2026-05-11T15:53:32,006 adding 'lm_eval/tasks/galician_bench/truthfulqa_gl_gen.yaml' 2026-05-11T15:53:32,007 adding 'lm_eval/tasks/galician_bench/truthfulqa_gl_mc1.yaml' 2026-05-11T15:53:32,008 adding 'lm_eval/tasks/galician_bench/truthfulqa_gl_mc2.yaml' 2026-05-11T15:53:32,010 adding 'lm_eval/tasks/galician_bench/utils.py' 2026-05-11T15:53:32,012 adding 'lm_eval/tasks/galician_bench/xnli_gl.yaml' 2026-05-11T15:53:32,013 adding 'lm_eval/tasks/galician_bench/xstorycloze_gl.yaml' 2026-05-11T15:53:32,015 adding 'lm_eval/tasks/galician_bench/flores_gl/_flores_common_yaml' 2026-05-11T15:53:32,016 adding 'lm_eval/tasks/galician_bench/flores_gl/create_yamls_flores_gl.py' 2026-05-11T15:53:32,018 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_ca-gl.yaml' 2026-05-11T15:53:32,019 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_de-gl.yaml' 2026-05-11T15:53:32,020 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_en-gl.yaml' 2026-05-11T15:53:32,021 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_es-gl.yaml' 2026-05-11T15:53:32,023 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_eu-gl.yaml' 2026-05-11T15:53:32,024 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_fr-gl.yaml' 2026-05-11T15:53:32,025 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-ca.yaml' 2026-05-11T15:53:32,026 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-de.yaml' 2026-05-11T15:53:32,028 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-en.yaml' 2026-05-11T15:53:32,029 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-es.yaml' 2026-05-11T15:53:32,030 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-eu.yaml' 2026-05-11T15:53:32,032 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-fr.yaml' 2026-05-11T15:53:32,033 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-it.yaml' 2026-05-11T15:53:32,034 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl-pt.yaml' 2026-05-11T15:53:32,035 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_gl.yaml' 2026-05-11T15:53:32,037 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_it-gl.yaml' 2026-05-11T15:53:32,038 adding 'lm_eval/tasks/galician_bench/flores_gl/flores_pt-gl.yaml' 2026-05-11T15:53:32,040 adding 'lm_eval/tasks/glianorex/README.md' 2026-05-11T15:53:32,041 adding 'lm_eval/tasks/glianorex/glianorex.yaml' 2026-05-11T15:53:32,043 adding 'lm_eval/tasks/glianorex/glianorex_en.yaml' 2026-05-11T15:53:32,044 adding 'lm_eval/tasks/glianorex/glianorex_fr.yaml' 2026-05-11T15:53:32,045 adding 'lm_eval/tasks/glianorex/preprocess_glianorex.py' 2026-05-11T15:53:32,047 adding 'lm_eval/tasks/global_mmlu/README.md' 2026-05-11T15:53:32,050 adding 'lm_eval/tasks/global_mmlu/default/ar/_ar_template_yaml' 2026-05-11T15:53:32,051 adding 'lm_eval/tasks/global_mmlu/default/ar/_global_mmlu_ar.yaml' 2026-05-11T15:53:32,052 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_business.yaml' 2026-05-11T15:53:32,053 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_humanities.yaml' 2026-05-11T15:53:32,054 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_medical.yaml' 2026-05-11T15:53:32,056 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_other.yaml' 2026-05-11T15:53:32,057 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_social_sciences.yaml' 2026-05-11T15:53:32,058 adding 'lm_eval/tasks/global_mmlu/default/ar/global_mmlu_ar_stem.yaml' 2026-05-11T15:53:32,059 adding 'lm_eval/tasks/global_mmlu/default/ar/utils.py' 2026-05-11T15:53:32,061 adding 'lm_eval/tasks/global_mmlu/default/bn/_bn_template_yaml' 2026-05-11T15:53:32,062 adding 'lm_eval/tasks/global_mmlu/default/bn/_global_mmlu_bn.yaml' 2026-05-11T15:53:32,063 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_business.yaml' 2026-05-11T15:53:32,064 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_humanities.yaml' 2026-05-11T15:53:32,066 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_medical.yaml' 2026-05-11T15:53:32,067 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_other.yaml' 2026-05-11T15:53:32,068 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_social_sciences.yaml' 2026-05-11T15:53:32,069 adding 'lm_eval/tasks/global_mmlu/default/bn/global_mmlu_bn_stem.yaml' 2026-05-11T15:53:32,070 adding 'lm_eval/tasks/global_mmlu/default/bn/utils.py' 2026-05-11T15:53:32,072 adding 'lm_eval/tasks/global_mmlu/default/de/_de_template_yaml' 2026-05-11T15:53:32,074 adding 'lm_eval/tasks/global_mmlu/default/de/_global_mmlu_de.yaml' 2026-05-11T15:53:32,075 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_business.yaml' 2026-05-11T15:53:32,076 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_humanities.yaml' 2026-05-11T15:53:32,077 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_medical.yaml' 2026-05-11T15:53:32,079 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_other.yaml' 2026-05-11T15:53:32,080 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_social_sciences.yaml' 2026-05-11T15:53:32,081 adding 'lm_eval/tasks/global_mmlu/default/de/global_mmlu_de_stem.yaml' 2026-05-11T15:53:32,082 adding 'lm_eval/tasks/global_mmlu/default/de/utils.py' 2026-05-11T15:53:32,084 adding 'lm_eval/tasks/global_mmlu/default/en/_en_template_yaml' 2026-05-11T15:53:32,086 adding 'lm_eval/tasks/global_mmlu/default/en/_global_mmlu_en.yaml' 2026-05-11T15:53:32,087 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_business.yaml' 2026-05-11T15:53:32,088 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_humanities.yaml' 2026-05-11T15:53:32,089 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_medical.yaml' 2026-05-11T15:53:32,091 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_other.yaml' 2026-05-11T15:53:32,092 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_social_sciences.yaml' 2026-05-11T15:53:32,093 adding 'lm_eval/tasks/global_mmlu/default/en/global_mmlu_en_stem.yaml' 2026-05-11T15:53:32,094 adding 'lm_eval/tasks/global_mmlu/default/en/utils.py' 2026-05-11T15:53:32,096 adding 'lm_eval/tasks/global_mmlu/default/es/_es_template_yaml' 2026-05-11T15:53:32,097 adding 'lm_eval/tasks/global_mmlu/default/es/_global_mmlu_es.yaml' 2026-05-11T15:53:32,099 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_business.yaml' 2026-05-11T15:53:32,100 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_humanities.yaml' 2026-05-11T15:53:32,101 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_medical.yaml' 2026-05-11T15:53:32,102 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_other.yaml' 2026-05-11T15:53:32,103 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_social_sciences.yaml' 2026-05-11T15:53:32,104 adding 'lm_eval/tasks/global_mmlu/default/es/global_mmlu_es_stem.yaml' 2026-05-11T15:53:32,105 adding 'lm_eval/tasks/global_mmlu/default/es/utils.py' 2026-05-11T15:53:32,107 adding 'lm_eval/tasks/global_mmlu/default/fr/_fr_template_yaml' 2026-05-11T15:53:32,109 adding 'lm_eval/tasks/global_mmlu/default/fr/_global_mmlu_fr.yaml' 2026-05-11T15:53:32,110 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_business.yaml' 2026-05-11T15:53:32,111 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_humanities.yaml' 2026-05-11T15:53:32,112 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_medical.yaml' 2026-05-11T15:53:32,114 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_other.yaml' 2026-05-11T15:53:32,115 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_social_sciences.yaml' 2026-05-11T15:53:32,116 adding 'lm_eval/tasks/global_mmlu/default/fr/global_mmlu_fr_stem.yaml' 2026-05-11T15:53:32,117 adding 'lm_eval/tasks/global_mmlu/default/fr/utils.py' 2026-05-11T15:53:32,118 adding 'lm_eval/tasks/global_mmlu/default/hi/_global_mmlu_hi.yaml' 2026-05-11T15:53:32,120 adding 'lm_eval/tasks/global_mmlu/default/hi/_hi_template_yaml' 2026-05-11T15:53:32,121 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_business.yaml' 2026-05-11T15:53:32,122 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_humanities.yaml' 2026-05-11T15:53:32,123 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_medical.yaml' 2026-05-11T15:53:32,124 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_other.yaml' 2026-05-11T15:53:32,125 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_social_sciences.yaml' 2026-05-11T15:53:32,126 adding 'lm_eval/tasks/global_mmlu/default/hi/global_mmlu_hi_stem.yaml' 2026-05-11T15:53:32,128 adding 'lm_eval/tasks/global_mmlu/default/hi/utils.py' 2026-05-11T15:53:32,130 adding 'lm_eval/tasks/global_mmlu/default/id/_global_mmlu_id.yaml' 2026-05-11T15:53:32,131 adding 'lm_eval/tasks/global_mmlu/default/id/_id_template_yaml' 2026-05-11T15:53:32,132 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_business.yaml' 2026-05-11T15:53:32,133 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_humanities.yaml' 2026-05-11T15:53:32,134 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_medical.yaml' 2026-05-11T15:53:32,136 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_other.yaml' 2026-05-11T15:53:32,137 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_social_sciences.yaml' 2026-05-11T15:53:32,138 adding 'lm_eval/tasks/global_mmlu/default/id/global_mmlu_id_stem.yaml' 2026-05-11T15:53:32,139 adding 'lm_eval/tasks/global_mmlu/default/id/utils.py' 2026-05-11T15:53:32,141 adding 'lm_eval/tasks/global_mmlu/default/it/_global_mmlu_it.yaml' 2026-05-11T15:53:32,142 adding 'lm_eval/tasks/global_mmlu/default/it/_it_template_yaml' 2026-05-11T15:53:32,143 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_business.yaml' 2026-05-11T15:53:32,145 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_humanities.yaml' 2026-05-11T15:53:32,146 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_medical.yaml' 2026-05-11T15:53:32,147 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_other.yaml' 2026-05-11T15:53:32,148 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_social_sciences.yaml' 2026-05-11T15:53:32,149 adding 'lm_eval/tasks/global_mmlu/default/it/global_mmlu_it_stem.yaml' 2026-05-11T15:53:32,151 adding 'lm_eval/tasks/global_mmlu/default/it/utils.py' 2026-05-11T15:53:32,152 adding 'lm_eval/tasks/global_mmlu/default/ja/_global_mmlu_ja.yaml' 2026-05-11T15:53:32,153 adding 'lm_eval/tasks/global_mmlu/default/ja/_ja_template_yaml' 2026-05-11T15:53:32,155 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_business.yaml' 2026-05-11T15:53:32,156 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml' 2026-05-11T15:53:32,157 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml' 2026-05-11T15:53:32,158 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml' 2026-05-11T15:53:32,159 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml' 2026-05-11T15:53:32,160 adding 'lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml' 2026-05-11T15:53:32,161 adding 'lm_eval/tasks/global_mmlu/default/ja/utils.py' 2026-05-11T15:53:32,163 adding 'lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml' 2026-05-11T15:53:32,164 adding 'lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml' 2026-05-11T15:53:32,165 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml' 2026-05-11T15:53:32,166 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml' 2026-05-11T15:53:32,167 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml' 2026-05-11T15:53:32,168 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml' 2026-05-11T15:53:32,170 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml' 2026-05-11T15:53:32,171 adding 'lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml' 2026-05-11T15:53:32,172 adding 'lm_eval/tasks/global_mmlu/default/ko/utils.py' 2026-05-11T15:53:32,174 adding 'lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml' 2026-05-11T15:53:32,175 adding 'lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml' 2026-05-11T15:53:32,176 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml' 2026-05-11T15:53:32,177 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml' 2026-05-11T15:53:32,179 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml' 2026-05-11T15:53:32,180 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_other.yaml' 2026-05-11T15:53:32,181 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_social_sciences.yaml' 2026-05-11T15:53:32,182 adding 'lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_stem.yaml' 2026-05-11T15:53:32,184 adding 'lm_eval/tasks/global_mmlu/default/pt/utils.py' 2026-05-11T15:53:32,186 adding 'lm_eval/tasks/global_mmlu/default/sw/_global_mmlu_sw.yaml' 2026-05-11T15:53:32,187 adding 'lm_eval/tasks/global_mmlu/default/sw/_sw_template_yaml' 2026-05-11T15:53:32,188 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_business.yaml' 2026-05-11T15:53:32,189 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_humanities.yaml' 2026-05-11T15:53:32,190 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_medical.yaml' 2026-05-11T15:53:32,191 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_other.yaml' 2026-05-11T15:53:32,193 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_social_sciences.yaml' 2026-05-11T15:53:32,194 adding 'lm_eval/tasks/global_mmlu/default/sw/global_mmlu_sw_stem.yaml' 2026-05-11T15:53:32,195 adding 'lm_eval/tasks/global_mmlu/default/sw/utils.py' 2026-05-11T15:53:32,197 adding 'lm_eval/tasks/global_mmlu/default/yo/_global_mmlu_yo.yaml' 2026-05-11T15:53:32,198 adding 'lm_eval/tasks/global_mmlu/default/yo/_yo_template_yaml' 2026-05-11T15:53:32,199 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_business.yaml' 2026-05-11T15:53:32,201 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_humanities.yaml' 2026-05-11T15:53:32,202 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_medical.yaml' 2026-05-11T15:53:32,203 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_other.yaml' 2026-05-11T15:53:32,204 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_social_sciences.yaml' 2026-05-11T15:53:32,205 adding 'lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml' 2026-05-11T15:53:32,206 adding 'lm_eval/tasks/global_mmlu/default/yo/utils.py' 2026-05-11T15:53:32,208 adding 'lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml' 2026-05-11T15:53:32,209 adding 'lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml' 2026-05-11T15:53:32,210 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml' 2026-05-11T15:53:32,211 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml' 2026-05-11T15:53:32,213 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml' 2026-05-11T15:53:32,214 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml' 2026-05-11T15:53:32,215 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml' 2026-05-11T15:53:32,216 adding 'lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml' 2026-05-11T15:53:32,217 adding 'lm_eval/tasks/global_mmlu/default/zh/utils.py' 2026-05-11T15:53:32,221 adding 'lm_eval/tasks/global_mmlu/full/am/_am_template_yaml' 2026-05-11T15:53:32,222 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml' 2026-05-11T15:53:32,223 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_humanities.yaml' 2026-05-11T15:53:32,224 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_other.yaml' 2026-05-11T15:53:32,226 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_social_sciences.yaml' 2026-05-11T15:53:32,227 adding 'lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am_stem.yaml' 2026-05-11T15:53:32,228 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_abstract_algebra.yaml' 2026-05-11T15:53:32,229 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_anatomy.yaml' 2026-05-11T15:53:32,231 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_astronomy.yaml' 2026-05-11T15:53:32,232 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_business_ethics.yaml' 2026-05-11T15:53:32,233 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_clinical_knowledge.yaml' 2026-05-11T15:53:32,234 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_biology.yaml' 2026-05-11T15:53:32,235 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_chemistry.yaml' 2026-05-11T15:53:32,237 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_computer_science.yaml' 2026-05-11T15:53:32,238 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_mathematics.yaml' 2026-05-11T15:53:32,239 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_medicine.yaml' 2026-05-11T15:53:32,240 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_college_physics.yaml' 2026-05-11T15:53:32,241 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_computer_security.yaml' 2026-05-11T15:53:32,243 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_conceptual_physics.yaml' 2026-05-11T15:53:32,244 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_econometrics.yaml' 2026-05-11T15:53:32,245 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_electrical_engineering.yaml' 2026-05-11T15:53:32,246 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_elementary_mathematics.yaml' 2026-05-11T15:53:32,247 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_formal_logic.yaml' 2026-05-11T15:53:32,249 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_global_facts.yaml' 2026-05-11T15:53:32,250 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_biology.yaml' 2026-05-11T15:53:32,251 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_chemistry.yaml' 2026-05-11T15:53:32,252 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_computer_science.yaml' 2026-05-11T15:53:32,253 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_european_history.yaml' 2026-05-11T15:53:32,255 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_geography.yaml' 2026-05-11T15:53:32,256 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,257 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,258 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_mathematics.yaml' 2026-05-11T15:53:32,259 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_microeconomics.yaml' 2026-05-11T15:53:32,260 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_physics.yaml' 2026-05-11T15:53:32,262 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_psychology.yaml' 2026-05-11T15:53:32,263 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_statistics.yaml' 2026-05-11T15:53:32,264 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_us_history.yaml' 2026-05-11T15:53:32,265 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_high_school_world_history.yaml' 2026-05-11T15:53:32,267 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_aging.yaml' 2026-05-11T15:53:32,268 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_human_sexuality.yaml' 2026-05-11T15:53:32,269 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_international_law.yaml' 2026-05-11T15:53:32,270 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_jurisprudence.yaml' 2026-05-11T15:53:32,272 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_logical_fallacies.yaml' 2026-05-11T15:53:32,273 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_machine_learning.yaml' 2026-05-11T15:53:32,275 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_management.yaml' 2026-05-11T15:53:32,277 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_marketing.yaml' 2026-05-11T15:53:32,278 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_medical_genetics.yaml' 2026-05-11T15:53:32,279 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_miscellaneous.yaml' 2026-05-11T15:53:32,282 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_disputes.yaml' 2026-05-11T15:53:32,283 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_moral_scenarios.yaml' 2026-05-11T15:53:32,284 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_nutrition.yaml' 2026-05-11T15:53:32,285 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_philosophy.yaml' 2026-05-11T15:53:32,286 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_prehistory.yaml' 2026-05-11T15:53:32,288 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_accounting.yaml' 2026-05-11T15:53:32,289 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_law.yaml' 2026-05-11T15:53:32,290 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_medicine.yaml' 2026-05-11T15:53:32,291 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_professional_psychology.yaml' 2026-05-11T15:53:32,292 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_public_relations.yaml' 2026-05-11T15:53:32,294 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_security_studies.yaml' 2026-05-11T15:53:32,295 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_sociology.yaml' 2026-05-11T15:53:32,296 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_us_foreign_policy.yaml' 2026-05-11T15:53:32,297 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_virology.yaml' 2026-05-11T15:53:32,299 adding 'lm_eval/tasks/global_mmlu/full/am/global_mmlu_full_am_world_religions.yaml' 2026-05-11T15:53:32,300 adding 'lm_eval/tasks/global_mmlu/full/am/utils.py' 2026-05-11T15:53:32,303 adding 'lm_eval/tasks/global_mmlu/full/ar/_ar_template_yaml' 2026-05-11T15:53:32,304 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml' 2026-05-11T15:53:32,305 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_humanities.yaml' 2026-05-11T15:53:32,307 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_other.yaml' 2026-05-11T15:53:32,308 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_social_sciences.yaml' 2026-05-11T15:53:32,309 adding 'lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar_stem.yaml' 2026-05-11T15:53:32,310 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_abstract_algebra.yaml' 2026-05-11T15:53:32,311 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_anatomy.yaml' 2026-05-11T15:53:32,312 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_astronomy.yaml' 2026-05-11T15:53:32,313 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_business_ethics.yaml' 2026-05-11T15:53:32,315 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_clinical_knowledge.yaml' 2026-05-11T15:53:32,316 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_biology.yaml' 2026-05-11T15:53:32,317 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_chemistry.yaml' 2026-05-11T15:53:32,318 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_computer_science.yaml' 2026-05-11T15:53:32,320 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_mathematics.yaml' 2026-05-11T15:53:32,321 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_medicine.yaml' 2026-05-11T15:53:32,322 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_college_physics.yaml' 2026-05-11T15:53:32,323 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_computer_security.yaml' 2026-05-11T15:53:32,324 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_conceptual_physics.yaml' 2026-05-11T15:53:32,325 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_econometrics.yaml' 2026-05-11T15:53:32,327 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_electrical_engineering.yaml' 2026-05-11T15:53:32,328 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_elementary_mathematics.yaml' 2026-05-11T15:53:32,329 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_formal_logic.yaml' 2026-05-11T15:53:32,330 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_global_facts.yaml' 2026-05-11T15:53:32,332 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_biology.yaml' 2026-05-11T15:53:32,333 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_chemistry.yaml' 2026-05-11T15:53:32,334 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_computer_science.yaml' 2026-05-11T15:53:32,335 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_european_history.yaml' 2026-05-11T15:53:32,337 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_geography.yaml' 2026-05-11T15:53:32,338 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,339 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,341 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_mathematics.yaml' 2026-05-11T15:53:32,342 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_microeconomics.yaml' 2026-05-11T15:53:32,343 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_physics.yaml' 2026-05-11T15:53:32,344 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_psychology.yaml' 2026-05-11T15:53:32,346 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_statistics.yaml' 2026-05-11T15:53:32,347 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_us_history.yaml' 2026-05-11T15:53:32,348 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_high_school_world_history.yaml' 2026-05-11T15:53:32,349 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_aging.yaml' 2026-05-11T15:53:32,350 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_human_sexuality.yaml' 2026-05-11T15:53:32,352 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_international_law.yaml' 2026-05-11T15:53:32,353 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_jurisprudence.yaml' 2026-05-11T15:53:32,354 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_logical_fallacies.yaml' 2026-05-11T15:53:32,355 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_machine_learning.yaml' 2026-05-11T15:53:32,356 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_management.yaml' 2026-05-11T15:53:32,358 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_marketing.yaml' 2026-05-11T15:53:32,359 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_medical_genetics.yaml' 2026-05-11T15:53:32,360 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_miscellaneous.yaml' 2026-05-11T15:53:32,361 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_disputes.yaml' 2026-05-11T15:53:32,363 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_moral_scenarios.yaml' 2026-05-11T15:53:32,364 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_nutrition.yaml' 2026-05-11T15:53:32,365 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_philosophy.yaml' 2026-05-11T15:53:32,366 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_prehistory.yaml' 2026-05-11T15:53:32,367 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_accounting.yaml' 2026-05-11T15:53:32,369 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_law.yaml' 2026-05-11T15:53:32,370 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_medicine.yaml' 2026-05-11T15:53:32,371 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_professional_psychology.yaml' 2026-05-11T15:53:32,372 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_public_relations.yaml' 2026-05-11T15:53:32,373 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_security_studies.yaml' 2026-05-11T15:53:32,374 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_sociology.yaml' 2026-05-11T15:53:32,376 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_us_foreign_policy.yaml' 2026-05-11T15:53:32,377 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_virology.yaml' 2026-05-11T15:53:32,378 adding 'lm_eval/tasks/global_mmlu/full/ar/global_mmlu_full_ar_world_religions.yaml' 2026-05-11T15:53:32,379 adding 'lm_eval/tasks/global_mmlu/full/ar/utils.py' 2026-05-11T15:53:32,382 adding 'lm_eval/tasks/global_mmlu/full/bn/_bn_template_yaml' 2026-05-11T15:53:32,384 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml' 2026-05-11T15:53:32,385 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_humanities.yaml' 2026-05-11T15:53:32,386 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_other.yaml' 2026-05-11T15:53:32,387 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_social_sciences.yaml' 2026-05-11T15:53:32,389 adding 'lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn_stem.yaml' 2026-05-11T15:53:32,390 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_abstract_algebra.yaml' 2026-05-11T15:53:32,391 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_anatomy.yaml' 2026-05-11T15:53:32,392 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_astronomy.yaml' 2026-05-11T15:53:32,394 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_business_ethics.yaml' 2026-05-11T15:53:32,395 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_clinical_knowledge.yaml' 2026-05-11T15:53:32,396 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_biology.yaml' 2026-05-11T15:53:32,398 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_chemistry.yaml' 2026-05-11T15:53:32,399 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_computer_science.yaml' 2026-05-11T15:53:32,400 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_mathematics.yaml' 2026-05-11T15:53:32,401 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_medicine.yaml' 2026-05-11T15:53:32,402 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_college_physics.yaml' 2026-05-11T15:53:32,403 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_computer_security.yaml' 2026-05-11T15:53:32,405 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_conceptual_physics.yaml' 2026-05-11T15:53:32,406 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_econometrics.yaml' 2026-05-11T15:53:32,407 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_electrical_engineering.yaml' 2026-05-11T15:53:32,408 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_elementary_mathematics.yaml' 2026-05-11T15:53:32,410 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_formal_logic.yaml' 2026-05-11T15:53:32,411 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_global_facts.yaml' 2026-05-11T15:53:32,412 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_biology.yaml' 2026-05-11T15:53:32,413 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_chemistry.yaml' 2026-05-11T15:53:32,414 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_computer_science.yaml' 2026-05-11T15:53:32,416 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_european_history.yaml' 2026-05-11T15:53:32,417 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_geography.yaml' 2026-05-11T15:53:32,418 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,419 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,420 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_mathematics.yaml' 2026-05-11T15:53:32,421 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_microeconomics.yaml' 2026-05-11T15:53:32,423 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_physics.yaml' 2026-05-11T15:53:32,424 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_psychology.yaml' 2026-05-11T15:53:32,425 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_statistics.yaml' 2026-05-11T15:53:32,426 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_us_history.yaml' 2026-05-11T15:53:32,427 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_high_school_world_history.yaml' 2026-05-11T15:53:32,429 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_aging.yaml' 2026-05-11T15:53:32,430 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_human_sexuality.yaml' 2026-05-11T15:53:32,431 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_international_law.yaml' 2026-05-11T15:53:32,433 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_jurisprudence.yaml' 2026-05-11T15:53:32,434 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_logical_fallacies.yaml' 2026-05-11T15:53:32,435 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_machine_learning.yaml' 2026-05-11T15:53:32,436 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_management.yaml' 2026-05-11T15:53:32,438 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_marketing.yaml' 2026-05-11T15:53:32,439 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_medical_genetics.yaml' 2026-05-11T15:53:32,440 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_miscellaneous.yaml' 2026-05-11T15:53:32,441 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_disputes.yaml' 2026-05-11T15:53:32,442 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_moral_scenarios.yaml' 2026-05-11T15:53:32,443 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_nutrition.yaml' 2026-05-11T15:53:32,444 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_philosophy.yaml' 2026-05-11T15:53:32,445 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_prehistory.yaml' 2026-05-11T15:53:32,447 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_accounting.yaml' 2026-05-11T15:53:32,448 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_law.yaml' 2026-05-11T15:53:32,449 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_medicine.yaml' 2026-05-11T15:53:32,450 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_professional_psychology.yaml' 2026-05-11T15:53:32,451 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_public_relations.yaml' 2026-05-11T15:53:32,453 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_security_studies.yaml' 2026-05-11T15:53:32,454 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_sociology.yaml' 2026-05-11T15:53:32,455 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_us_foreign_policy.yaml' 2026-05-11T15:53:32,456 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_virology.yaml' 2026-05-11T15:53:32,457 adding 'lm_eval/tasks/global_mmlu/full/bn/global_mmlu_full_bn_world_religions.yaml' 2026-05-11T15:53:32,458 adding 'lm_eval/tasks/global_mmlu/full/bn/utils.py' 2026-05-11T15:53:32,462 adding 'lm_eval/tasks/global_mmlu/full/cs/_cs_template_yaml' 2026-05-11T15:53:32,463 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml' 2026-05-11T15:53:32,464 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_humanities.yaml' 2026-05-11T15:53:32,465 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_other.yaml' 2026-05-11T15:53:32,467 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_social_sciences.yaml' 2026-05-11T15:53:32,468 adding 'lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs_stem.yaml' 2026-05-11T15:53:32,469 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_abstract_algebra.yaml' 2026-05-11T15:53:32,470 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_anatomy.yaml' 2026-05-11T15:53:32,472 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_astronomy.yaml' 2026-05-11T15:53:32,473 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_business_ethics.yaml' 2026-05-11T15:53:32,474 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_clinical_knowledge.yaml' 2026-05-11T15:53:32,475 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_biology.yaml' 2026-05-11T15:53:32,476 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_chemistry.yaml' 2026-05-11T15:53:32,478 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_computer_science.yaml' 2026-05-11T15:53:32,479 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_mathematics.yaml' 2026-05-11T15:53:32,480 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_medicine.yaml' 2026-05-11T15:53:32,482 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_college_physics.yaml' 2026-05-11T15:53:32,483 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_computer_security.yaml' 2026-05-11T15:53:32,484 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_conceptual_physics.yaml' 2026-05-11T15:53:32,486 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_econometrics.yaml' 2026-05-11T15:53:32,487 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_electrical_engineering.yaml' 2026-05-11T15:53:32,488 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_elementary_mathematics.yaml' 2026-05-11T15:53:32,489 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_formal_logic.yaml' 2026-05-11T15:53:32,490 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_global_facts.yaml' 2026-05-11T15:53:32,491 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_biology.yaml' 2026-05-11T15:53:32,492 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_chemistry.yaml' 2026-05-11T15:53:32,494 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_computer_science.yaml' 2026-05-11T15:53:32,495 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_european_history.yaml' 2026-05-11T15:53:32,496 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_geography.yaml' 2026-05-11T15:53:32,497 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,498 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,500 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_mathematics.yaml' 2026-05-11T15:53:32,501 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_microeconomics.yaml' 2026-05-11T15:53:32,502 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_physics.yaml' 2026-05-11T15:53:32,503 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_psychology.yaml' 2026-05-11T15:53:32,504 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_statistics.yaml' 2026-05-11T15:53:32,505 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_us_history.yaml' 2026-05-11T15:53:32,506 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_high_school_world_history.yaml' 2026-05-11T15:53:32,508 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_aging.yaml' 2026-05-11T15:53:32,509 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_human_sexuality.yaml' 2026-05-11T15:53:32,510 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_international_law.yaml' 2026-05-11T15:53:32,511 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_jurisprudence.yaml' 2026-05-11T15:53:32,512 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_logical_fallacies.yaml' 2026-05-11T15:53:32,513 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_machine_learning.yaml' 2026-05-11T15:53:32,515 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_management.yaml' 2026-05-11T15:53:32,516 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_marketing.yaml' 2026-05-11T15:53:32,517 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_medical_genetics.yaml' 2026-05-11T15:53:32,518 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_miscellaneous.yaml' 2026-05-11T15:53:32,519 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_disputes.yaml' 2026-05-11T15:53:32,520 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_moral_scenarios.yaml' 2026-05-11T15:53:32,522 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_nutrition.yaml' 2026-05-11T15:53:32,523 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_philosophy.yaml' 2026-05-11T15:53:32,524 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_prehistory.yaml' 2026-05-11T15:53:32,525 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_accounting.yaml' 2026-05-11T15:53:32,526 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_law.yaml' 2026-05-11T15:53:32,527 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_medicine.yaml' 2026-05-11T15:53:32,529 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_professional_psychology.yaml' 2026-05-11T15:53:32,530 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_public_relations.yaml' 2026-05-11T15:53:32,531 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_security_studies.yaml' 2026-05-11T15:53:32,532 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_sociology.yaml' 2026-05-11T15:53:32,533 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_us_foreign_policy.yaml' 2026-05-11T15:53:32,534 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_virology.yaml' 2026-05-11T15:53:32,535 adding 'lm_eval/tasks/global_mmlu/full/cs/global_mmlu_full_cs_world_religions.yaml' 2026-05-11T15:53:32,537 adding 'lm_eval/tasks/global_mmlu/full/cs/utils.py' 2026-05-11T15:53:32,539 adding 'lm_eval/tasks/global_mmlu/full/de/_de_template_yaml' 2026-05-11T15:53:32,540 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml' 2026-05-11T15:53:32,542 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_humanities.yaml' 2026-05-11T15:53:32,543 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_other.yaml' 2026-05-11T15:53:32,544 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_social_sciences.yaml' 2026-05-11T15:53:32,545 adding 'lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de_stem.yaml' 2026-05-11T15:53:32,546 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_abstract_algebra.yaml' 2026-05-11T15:53:32,547 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_anatomy.yaml' 2026-05-11T15:53:32,548 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_astronomy.yaml' 2026-05-11T15:53:32,550 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_business_ethics.yaml' 2026-05-11T15:53:32,551 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_clinical_knowledge.yaml' 2026-05-11T15:53:32,552 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_biology.yaml' 2026-05-11T15:53:32,553 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_chemistry.yaml' 2026-05-11T15:53:32,554 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_computer_science.yaml' 2026-05-11T15:53:32,555 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_mathematics.yaml' 2026-05-11T15:53:32,557 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_medicine.yaml' 2026-05-11T15:53:32,558 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_college_physics.yaml' 2026-05-11T15:53:32,559 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_computer_security.yaml' 2026-05-11T15:53:32,560 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_conceptual_physics.yaml' 2026-05-11T15:53:32,561 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_econometrics.yaml' 2026-05-11T15:53:32,562 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_electrical_engineering.yaml' 2026-05-11T15:53:32,563 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_elementary_mathematics.yaml' 2026-05-11T15:53:32,565 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_formal_logic.yaml' 2026-05-11T15:53:32,566 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_global_facts.yaml' 2026-05-11T15:53:32,567 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_biology.yaml' 2026-05-11T15:53:32,568 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_chemistry.yaml' 2026-05-11T15:53:32,569 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_computer_science.yaml' 2026-05-11T15:53:32,570 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_european_history.yaml' 2026-05-11T15:53:32,572 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_geography.yaml' 2026-05-11T15:53:32,573 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,574 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,575 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_mathematics.yaml' 2026-05-11T15:53:32,576 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_microeconomics.yaml' 2026-05-11T15:53:32,577 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_physics.yaml' 2026-05-11T15:53:32,578 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_psychology.yaml' 2026-05-11T15:53:32,579 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_statistics.yaml' 2026-05-11T15:53:32,580 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_us_history.yaml' 2026-05-11T15:53:32,582 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_high_school_world_history.yaml' 2026-05-11T15:53:32,583 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_aging.yaml' 2026-05-11T15:53:32,584 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_human_sexuality.yaml' 2026-05-11T15:53:32,585 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_international_law.yaml' 2026-05-11T15:53:32,586 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_jurisprudence.yaml' 2026-05-11T15:53:32,587 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_logical_fallacies.yaml' 2026-05-11T15:53:32,588 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_machine_learning.yaml' 2026-05-11T15:53:32,590 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_management.yaml' 2026-05-11T15:53:32,591 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_marketing.yaml' 2026-05-11T15:53:32,592 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_medical_genetics.yaml' 2026-05-11T15:53:32,593 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_miscellaneous.yaml' 2026-05-11T15:53:32,594 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_disputes.yaml' 2026-05-11T15:53:32,595 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_moral_scenarios.yaml' 2026-05-11T15:53:32,597 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_nutrition.yaml' 2026-05-11T15:53:32,598 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_philosophy.yaml' 2026-05-11T15:53:32,599 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_prehistory.yaml' 2026-05-11T15:53:32,600 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_accounting.yaml' 2026-05-11T15:53:32,601 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_law.yaml' 2026-05-11T15:53:32,602 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_medicine.yaml' 2026-05-11T15:53:32,604 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_professional_psychology.yaml' 2026-05-11T15:53:32,605 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_public_relations.yaml' 2026-05-11T15:53:32,606 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_security_studies.yaml' 2026-05-11T15:53:32,607 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_sociology.yaml' 2026-05-11T15:53:32,608 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_us_foreign_policy.yaml' 2026-05-11T15:53:32,610 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_virology.yaml' 2026-05-11T15:53:32,611 adding 'lm_eval/tasks/global_mmlu/full/de/global_mmlu_full_de_world_religions.yaml' 2026-05-11T15:53:32,612 adding 'lm_eval/tasks/global_mmlu/full/de/utils.py' 2026-05-11T15:53:32,615 adding 'lm_eval/tasks/global_mmlu/full/el/_el_template_yaml' 2026-05-11T15:53:32,617 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml' 2026-05-11T15:53:32,618 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_humanities.yaml' 2026-05-11T15:53:32,619 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_other.yaml' 2026-05-11T15:53:32,620 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_social_sciences.yaml' 2026-05-11T15:53:32,621 adding 'lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el_stem.yaml' 2026-05-11T15:53:32,623 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_abstract_algebra.yaml' 2026-05-11T15:53:32,624 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_anatomy.yaml' 2026-05-11T15:53:32,625 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_astronomy.yaml' 2026-05-11T15:53:32,626 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_business_ethics.yaml' 2026-05-11T15:53:32,627 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_clinical_knowledge.yaml' 2026-05-11T15:53:32,629 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_biology.yaml' 2026-05-11T15:53:32,630 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_chemistry.yaml' 2026-05-11T15:53:32,631 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_computer_science.yaml' 2026-05-11T15:53:32,632 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_mathematics.yaml' 2026-05-11T15:53:32,633 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_medicine.yaml' 2026-05-11T15:53:32,635 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_college_physics.yaml' 2026-05-11T15:53:32,636 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_computer_security.yaml' 2026-05-11T15:53:32,637 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_conceptual_physics.yaml' 2026-05-11T15:53:32,638 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_econometrics.yaml' 2026-05-11T15:53:32,639 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_electrical_engineering.yaml' 2026-05-11T15:53:32,640 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_elementary_mathematics.yaml' 2026-05-11T15:53:32,642 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_formal_logic.yaml' 2026-05-11T15:53:32,643 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_global_facts.yaml' 2026-05-11T15:53:32,644 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_biology.yaml' 2026-05-11T15:53:32,646 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_chemistry.yaml' 2026-05-11T15:53:32,647 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_computer_science.yaml' 2026-05-11T15:53:32,648 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_european_history.yaml' 2026-05-11T15:53:32,649 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_geography.yaml' 2026-05-11T15:53:32,651 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,652 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,653 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_mathematics.yaml' 2026-05-11T15:53:32,654 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_microeconomics.yaml' 2026-05-11T15:53:32,655 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_physics.yaml' 2026-05-11T15:53:32,657 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_psychology.yaml' 2026-05-11T15:53:32,658 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_statistics.yaml' 2026-05-11T15:53:32,659 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_us_history.yaml' 2026-05-11T15:53:32,660 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_high_school_world_history.yaml' 2026-05-11T15:53:32,661 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_aging.yaml' 2026-05-11T15:53:32,662 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_human_sexuality.yaml' 2026-05-11T15:53:32,663 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_international_law.yaml' 2026-05-11T15:53:32,664 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_jurisprudence.yaml' 2026-05-11T15:53:32,666 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_logical_fallacies.yaml' 2026-05-11T15:53:32,667 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_machine_learning.yaml' 2026-05-11T15:53:32,668 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_management.yaml' 2026-05-11T15:53:32,669 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_marketing.yaml' 2026-05-11T15:53:32,670 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_medical_genetics.yaml' 2026-05-11T15:53:32,671 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_miscellaneous.yaml' 2026-05-11T15:53:32,672 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_disputes.yaml' 2026-05-11T15:53:32,674 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_moral_scenarios.yaml' 2026-05-11T15:53:32,675 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_nutrition.yaml' 2026-05-11T15:53:32,676 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_philosophy.yaml' 2026-05-11T15:53:32,677 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_prehistory.yaml' 2026-05-11T15:53:32,678 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_accounting.yaml' 2026-05-11T15:53:32,679 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_law.yaml' 2026-05-11T15:53:32,681 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_medicine.yaml' 2026-05-11T15:53:32,682 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_professional_psychology.yaml' 2026-05-11T15:53:32,683 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_public_relations.yaml' 2026-05-11T15:53:32,684 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_security_studies.yaml' 2026-05-11T15:53:32,685 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_sociology.yaml' 2026-05-11T15:53:32,686 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_us_foreign_policy.yaml' 2026-05-11T15:53:32,688 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_virology.yaml' 2026-05-11T15:53:32,689 adding 'lm_eval/tasks/global_mmlu/full/el/global_mmlu_full_el_world_religions.yaml' 2026-05-11T15:53:32,690 adding 'lm_eval/tasks/global_mmlu/full/el/utils.py' 2026-05-11T15:53:32,693 adding 'lm_eval/tasks/global_mmlu/full/en/_en_template_yaml' 2026-05-11T15:53:32,695 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml' 2026-05-11T15:53:32,696 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_humanities.yaml' 2026-05-11T15:53:32,697 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_other.yaml' 2026-05-11T15:53:32,698 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_social_sciences.yaml' 2026-05-11T15:53:32,699 adding 'lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en_stem.yaml' 2026-05-11T15:53:32,701 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_abstract_algebra.yaml' 2026-05-11T15:53:32,702 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_anatomy.yaml' 2026-05-11T15:53:32,703 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_astronomy.yaml' 2026-05-11T15:53:32,704 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_business_ethics.yaml' 2026-05-11T15:53:32,705 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_clinical_knowledge.yaml' 2026-05-11T15:53:32,707 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_biology.yaml' 2026-05-11T15:53:32,708 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_chemistry.yaml' 2026-05-11T15:53:32,709 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_computer_science.yaml' 2026-05-11T15:53:32,710 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_mathematics.yaml' 2026-05-11T15:53:32,711 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_medicine.yaml' 2026-05-11T15:53:32,712 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_college_physics.yaml' 2026-05-11T15:53:32,713 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_computer_security.yaml' 2026-05-11T15:53:32,714 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_conceptual_physics.yaml' 2026-05-11T15:53:32,715 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_econometrics.yaml' 2026-05-11T15:53:32,716 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_electrical_engineering.yaml' 2026-05-11T15:53:32,718 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_elementary_mathematics.yaml' 2026-05-11T15:53:32,719 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_formal_logic.yaml' 2026-05-11T15:53:32,720 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_global_facts.yaml' 2026-05-11T15:53:32,721 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_biology.yaml' 2026-05-11T15:53:32,722 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_chemistry.yaml' 2026-05-11T15:53:32,723 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_computer_science.yaml' 2026-05-11T15:53:32,724 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_european_history.yaml' 2026-05-11T15:53:32,726 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_geography.yaml' 2026-05-11T15:53:32,727 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,728 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,729 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_mathematics.yaml' 2026-05-11T15:53:32,730 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_microeconomics.yaml' 2026-05-11T15:53:32,731 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_physics.yaml' 2026-05-11T15:53:32,733 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_psychology.yaml' 2026-05-11T15:53:32,734 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_statistics.yaml' 2026-05-11T15:53:32,735 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_us_history.yaml' 2026-05-11T15:53:32,736 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_high_school_world_history.yaml' 2026-05-11T15:53:32,737 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_aging.yaml' 2026-05-11T15:53:32,739 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_human_sexuality.yaml' 2026-05-11T15:53:32,740 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_international_law.yaml' 2026-05-11T15:53:32,741 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_jurisprudence.yaml' 2026-05-11T15:53:32,743 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_logical_fallacies.yaml' 2026-05-11T15:53:32,744 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_machine_learning.yaml' 2026-05-11T15:53:32,745 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_management.yaml' 2026-05-11T15:53:32,746 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_marketing.yaml' 2026-05-11T15:53:32,748 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_medical_genetics.yaml' 2026-05-11T15:53:32,749 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_miscellaneous.yaml' 2026-05-11T15:53:32,750 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_disputes.yaml' 2026-05-11T15:53:32,752 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_moral_scenarios.yaml' 2026-05-11T15:53:32,753 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_nutrition.yaml' 2026-05-11T15:53:32,754 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_philosophy.yaml' 2026-05-11T15:53:32,755 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_prehistory.yaml' 2026-05-11T15:53:32,756 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_accounting.yaml' 2026-05-11T15:53:32,757 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_law.yaml' 2026-05-11T15:53:32,759 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_medicine.yaml' 2026-05-11T15:53:32,760 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_professional_psychology.yaml' 2026-05-11T15:53:32,761 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_public_relations.yaml' 2026-05-11T15:53:32,762 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_security_studies.yaml' 2026-05-11T15:53:32,764 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_sociology.yaml' 2026-05-11T15:53:32,765 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_us_foreign_policy.yaml' 2026-05-11T15:53:32,766 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_virology.yaml' 2026-05-11T15:53:32,767 adding 'lm_eval/tasks/global_mmlu/full/en/global_mmlu_full_en_world_religions.yaml' 2026-05-11T15:53:32,769 adding 'lm_eval/tasks/global_mmlu/full/en/utils.py' 2026-05-11T15:53:32,772 adding 'lm_eval/tasks/global_mmlu/full/es/_es_template_yaml' 2026-05-11T15:53:32,773 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml' 2026-05-11T15:53:32,774 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_humanities.yaml' 2026-05-11T15:53:32,776 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_other.yaml' 2026-05-11T15:53:32,777 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_social_sciences.yaml' 2026-05-11T15:53:32,778 adding 'lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es_stem.yaml' 2026-05-11T15:53:32,779 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_abstract_algebra.yaml' 2026-05-11T15:53:32,780 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_anatomy.yaml' 2026-05-11T15:53:32,781 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_astronomy.yaml' 2026-05-11T15:53:32,783 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_business_ethics.yaml' 2026-05-11T15:53:32,784 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_clinical_knowledge.yaml' 2026-05-11T15:53:32,785 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_biology.yaml' 2026-05-11T15:53:32,786 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_chemistry.yaml' 2026-05-11T15:53:32,788 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_computer_science.yaml' 2026-05-11T15:53:32,789 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_mathematics.yaml' 2026-05-11T15:53:32,790 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_medicine.yaml' 2026-05-11T15:53:32,791 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_college_physics.yaml' 2026-05-11T15:53:32,792 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_computer_security.yaml' 2026-05-11T15:53:32,794 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_conceptual_physics.yaml' 2026-05-11T15:53:32,795 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_econometrics.yaml' 2026-05-11T15:53:32,796 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_electrical_engineering.yaml' 2026-05-11T15:53:32,797 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_elementary_mathematics.yaml' 2026-05-11T15:53:32,798 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_formal_logic.yaml' 2026-05-11T15:53:32,800 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_global_facts.yaml' 2026-05-11T15:53:32,801 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_biology.yaml' 2026-05-11T15:53:32,802 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_chemistry.yaml' 2026-05-11T15:53:32,803 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_computer_science.yaml' 2026-05-11T15:53:32,804 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_european_history.yaml' 2026-05-11T15:53:32,805 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_geography.yaml' 2026-05-11T15:53:32,806 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,808 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,809 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_mathematics.yaml' 2026-05-11T15:53:32,810 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_microeconomics.yaml' 2026-05-11T15:53:32,811 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_physics.yaml' 2026-05-11T15:53:32,812 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_psychology.yaml' 2026-05-11T15:53:32,813 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_statistics.yaml' 2026-05-11T15:53:32,814 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_us_history.yaml' 2026-05-11T15:53:32,816 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_high_school_world_history.yaml' 2026-05-11T15:53:32,817 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_aging.yaml' 2026-05-11T15:53:32,818 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_human_sexuality.yaml' 2026-05-11T15:53:32,819 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_international_law.yaml' 2026-05-11T15:53:32,820 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_jurisprudence.yaml' 2026-05-11T15:53:32,821 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_logical_fallacies.yaml' 2026-05-11T15:53:32,823 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_machine_learning.yaml' 2026-05-11T15:53:32,824 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_management.yaml' 2026-05-11T15:53:32,825 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_marketing.yaml' 2026-05-11T15:53:32,826 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_medical_genetics.yaml' 2026-05-11T15:53:32,827 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_miscellaneous.yaml' 2026-05-11T15:53:32,828 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_disputes.yaml' 2026-05-11T15:53:32,829 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_moral_scenarios.yaml' 2026-05-11T15:53:32,831 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_nutrition.yaml' 2026-05-11T15:53:32,832 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_philosophy.yaml' 2026-05-11T15:53:32,833 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_prehistory.yaml' 2026-05-11T15:53:32,834 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_accounting.yaml' 2026-05-11T15:53:32,835 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_law.yaml' 2026-05-11T15:53:32,837 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_medicine.yaml' 2026-05-11T15:53:32,838 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_professional_psychology.yaml' 2026-05-11T15:53:32,839 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_public_relations.yaml' 2026-05-11T15:53:32,841 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_security_studies.yaml' 2026-05-11T15:53:32,842 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_sociology.yaml' 2026-05-11T15:53:32,843 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_us_foreign_policy.yaml' 2026-05-11T15:53:32,844 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_virology.yaml' 2026-05-11T15:53:32,845 adding 'lm_eval/tasks/global_mmlu/full/es/global_mmlu_full_es_world_religions.yaml' 2026-05-11T15:53:32,846 adding 'lm_eval/tasks/global_mmlu/full/es/utils.py' 2026-05-11T15:53:32,850 adding 'lm_eval/tasks/global_mmlu/full/fa/_fa_template_yaml' 2026-05-11T15:53:32,851 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml' 2026-05-11T15:53:32,852 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_humanities.yaml' 2026-05-11T15:53:32,853 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_other.yaml' 2026-05-11T15:53:32,854 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_social_sciences.yaml' 2026-05-11T15:53:32,856 adding 'lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa_stem.yaml' 2026-05-11T15:53:32,857 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_abstract_algebra.yaml' 2026-05-11T15:53:32,858 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_anatomy.yaml' 2026-05-11T15:53:32,859 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_astronomy.yaml' 2026-05-11T15:53:32,860 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_business_ethics.yaml' 2026-05-11T15:53:32,861 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_clinical_knowledge.yaml' 2026-05-11T15:53:32,862 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_biology.yaml' 2026-05-11T15:53:32,863 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_chemistry.yaml' 2026-05-11T15:53:32,864 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_computer_science.yaml' 2026-05-11T15:53:32,866 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_mathematics.yaml' 2026-05-11T15:53:32,867 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_medicine.yaml' 2026-05-11T15:53:32,868 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_college_physics.yaml' 2026-05-11T15:53:32,869 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_computer_security.yaml' 2026-05-11T15:53:32,871 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_conceptual_physics.yaml' 2026-05-11T15:53:32,872 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_econometrics.yaml' 2026-05-11T15:53:32,873 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_electrical_engineering.yaml' 2026-05-11T15:53:32,874 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_elementary_mathematics.yaml' 2026-05-11T15:53:32,875 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_formal_logic.yaml' 2026-05-11T15:53:32,876 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_global_facts.yaml' 2026-05-11T15:53:32,878 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_biology.yaml' 2026-05-11T15:53:32,879 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_chemistry.yaml' 2026-05-11T15:53:32,880 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_computer_science.yaml' 2026-05-11T15:53:32,881 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_european_history.yaml' 2026-05-11T15:53:32,882 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_geography.yaml' 2026-05-11T15:53:32,884 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,885 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,886 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_mathematics.yaml' 2026-05-11T15:53:32,887 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_microeconomics.yaml' 2026-05-11T15:53:32,889 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_physics.yaml' 2026-05-11T15:53:32,890 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_psychology.yaml' 2026-05-11T15:53:32,891 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_statistics.yaml' 2026-05-11T15:53:32,892 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_us_history.yaml' 2026-05-11T15:53:32,893 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_high_school_world_history.yaml' 2026-05-11T15:53:32,894 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_aging.yaml' 2026-05-11T15:53:32,895 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_human_sexuality.yaml' 2026-05-11T15:53:32,896 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_international_law.yaml' 2026-05-11T15:53:32,897 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_jurisprudence.yaml' 2026-05-11T15:53:32,899 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_logical_fallacies.yaml' 2026-05-11T15:53:32,900 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_machine_learning.yaml' 2026-05-11T15:53:32,901 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_management.yaml' 2026-05-11T15:53:32,902 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_marketing.yaml' 2026-05-11T15:53:32,903 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_medical_genetics.yaml' 2026-05-11T15:53:32,904 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_miscellaneous.yaml' 2026-05-11T15:53:32,906 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_disputes.yaml' 2026-05-11T15:53:32,907 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_moral_scenarios.yaml' 2026-05-11T15:53:32,908 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_nutrition.yaml' 2026-05-11T15:53:32,909 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_philosophy.yaml' 2026-05-11T15:53:32,910 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_prehistory.yaml' 2026-05-11T15:53:32,911 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_accounting.yaml' 2026-05-11T15:53:32,913 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_law.yaml' 2026-05-11T15:53:32,914 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_medicine.yaml' 2026-05-11T15:53:32,915 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_professional_psychology.yaml' 2026-05-11T15:53:32,916 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_public_relations.yaml' 2026-05-11T15:53:32,917 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_security_studies.yaml' 2026-05-11T15:53:32,919 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_sociology.yaml' 2026-05-11T15:53:32,920 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_us_foreign_policy.yaml' 2026-05-11T15:53:32,921 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_virology.yaml' 2026-05-11T15:53:32,922 adding 'lm_eval/tasks/global_mmlu/full/fa/global_mmlu_full_fa_world_religions.yaml' 2026-05-11T15:53:32,923 adding 'lm_eval/tasks/global_mmlu/full/fa/utils.py' 2026-05-11T15:53:32,927 adding 'lm_eval/tasks/global_mmlu/full/fil/_fil_template_yaml' 2026-05-11T15:53:32,928 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil.yaml' 2026-05-11T15:53:32,929 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_humanities.yaml' 2026-05-11T15:53:32,930 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_other.yaml' 2026-05-11T15:53:32,931 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_social_sciences.yaml' 2026-05-11T15:53:32,933 adding 'lm_eval/tasks/global_mmlu/full/fil/_global_mmlu_full_fil_stem.yaml' 2026-05-11T15:53:32,934 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_abstract_algebra.yaml' 2026-05-11T15:53:32,935 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_anatomy.yaml' 2026-05-11T15:53:32,936 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_astronomy.yaml' 2026-05-11T15:53:32,937 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_business_ethics.yaml' 2026-05-11T15:53:32,939 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_clinical_knowledge.yaml' 2026-05-11T15:53:32,940 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_biology.yaml' 2026-05-11T15:53:32,941 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_chemistry.yaml' 2026-05-11T15:53:32,942 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_computer_science.yaml' 2026-05-11T15:53:32,943 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_mathematics.yaml' 2026-05-11T15:53:32,945 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_medicine.yaml' 2026-05-11T15:53:32,946 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_college_physics.yaml' 2026-05-11T15:53:32,947 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_computer_security.yaml' 2026-05-11T15:53:32,948 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_conceptual_physics.yaml' 2026-05-11T15:53:32,949 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_econometrics.yaml' 2026-05-11T15:53:32,951 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_electrical_engineering.yaml' 2026-05-11T15:53:32,952 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_elementary_mathematics.yaml' 2026-05-11T15:53:32,953 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_formal_logic.yaml' 2026-05-11T15:53:32,954 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_global_facts.yaml' 2026-05-11T15:53:32,955 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_biology.yaml' 2026-05-11T15:53:32,957 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_chemistry.yaml' 2026-05-11T15:53:32,958 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_computer_science.yaml' 2026-05-11T15:53:32,959 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_european_history.yaml' 2026-05-11T15:53:32,960 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_geography.yaml' 2026-05-11T15:53:32,962 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_government_and_politics.yaml' 2026-05-11T15:53:32,963 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_macroeconomics.yaml' 2026-05-11T15:53:32,964 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_mathematics.yaml' 2026-05-11T15:53:32,965 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_microeconomics.yaml' 2026-05-11T15:53:32,967 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_physics.yaml' 2026-05-11T15:53:32,968 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_psychology.yaml' 2026-05-11T15:53:32,969 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_statistics.yaml' 2026-05-11T15:53:32,970 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_us_history.yaml' 2026-05-11T15:53:32,971 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_high_school_world_history.yaml' 2026-05-11T15:53:32,973 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_aging.yaml' 2026-05-11T15:53:32,974 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_human_sexuality.yaml' 2026-05-11T15:53:32,975 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_international_law.yaml' 2026-05-11T15:53:32,976 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_jurisprudence.yaml' 2026-05-11T15:53:32,977 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_logical_fallacies.yaml' 2026-05-11T15:53:32,979 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_machine_learning.yaml' 2026-05-11T15:53:32,980 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_management.yaml' 2026-05-11T15:53:32,981 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_marketing.yaml' 2026-05-11T15:53:32,982 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_medical_genetics.yaml' 2026-05-11T15:53:32,983 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_miscellaneous.yaml' 2026-05-11T15:53:32,984 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_disputes.yaml' 2026-05-11T15:53:32,986 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_moral_scenarios.yaml' 2026-05-11T15:53:32,987 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_nutrition.yaml' 2026-05-11T15:53:32,988 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_philosophy.yaml' 2026-05-11T15:53:32,989 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_prehistory.yaml' 2026-05-11T15:53:32,990 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_accounting.yaml' 2026-05-11T15:53:32,991 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_law.yaml' 2026-05-11T15:53:32,993 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_medicine.yaml' 2026-05-11T15:53:32,994 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_professional_psychology.yaml' 2026-05-11T15:53:32,995 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_public_relations.yaml' 2026-05-11T15:53:32,997 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_security_studies.yaml' 2026-05-11T15:53:32,998 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_sociology.yaml' 2026-05-11T15:53:32,999 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_us_foreign_policy.yaml' 2026-05-11T15:53:33,000 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_virology.yaml' 2026-05-11T15:53:33,001 adding 'lm_eval/tasks/global_mmlu/full/fil/global_mmlu_full_fil_world_religions.yaml' 2026-05-11T15:53:33,003 adding 'lm_eval/tasks/global_mmlu/full/fil/utils.py' 2026-05-11T15:53:33,006 adding 'lm_eval/tasks/global_mmlu/full/fr/_fr_template_yaml' 2026-05-11T15:53:33,008 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr.yaml' 2026-05-11T15:53:33,009 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_humanities.yaml' 2026-05-11T15:53:33,010 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_other.yaml' 2026-05-11T15:53:33,011 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_social_sciences.yaml' 2026-05-11T15:53:33,013 adding 'lm_eval/tasks/global_mmlu/full/fr/_global_mmlu_full_fr_stem.yaml' 2026-05-11T15:53:33,014 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_abstract_algebra.yaml' 2026-05-11T15:53:33,015 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_anatomy.yaml' 2026-05-11T15:53:33,016 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_astronomy.yaml' 2026-05-11T15:53:33,018 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_business_ethics.yaml' 2026-05-11T15:53:33,019 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_clinical_knowledge.yaml' 2026-05-11T15:53:33,020 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_biology.yaml' 2026-05-11T15:53:33,021 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_chemistry.yaml' 2026-05-11T15:53:33,022 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_computer_science.yaml' 2026-05-11T15:53:33,024 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_mathematics.yaml' 2026-05-11T15:53:33,025 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_medicine.yaml' 2026-05-11T15:53:33,026 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_college_physics.yaml' 2026-05-11T15:53:33,027 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_computer_security.yaml' 2026-05-11T15:53:33,028 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_conceptual_physics.yaml' 2026-05-11T15:53:33,030 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_econometrics.yaml' 2026-05-11T15:53:33,031 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_electrical_engineering.yaml' 2026-05-11T15:53:33,032 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_elementary_mathematics.yaml' 2026-05-11T15:53:33,033 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_formal_logic.yaml' 2026-05-11T15:53:33,034 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_global_facts.yaml' 2026-05-11T15:53:33,035 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_biology.yaml' 2026-05-11T15:53:33,037 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_chemistry.yaml' 2026-05-11T15:53:33,038 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_computer_science.yaml' 2026-05-11T15:53:33,039 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_european_history.yaml' 2026-05-11T15:53:33,040 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_geography.yaml' 2026-05-11T15:53:33,041 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,043 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,044 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_mathematics.yaml' 2026-05-11T15:53:33,045 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_microeconomics.yaml' 2026-05-11T15:53:33,046 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_physics.yaml' 2026-05-11T15:53:33,047 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_psychology.yaml' 2026-05-11T15:53:33,049 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_statistics.yaml' 2026-05-11T15:53:33,050 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_us_history.yaml' 2026-05-11T15:53:33,051 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_high_school_world_history.yaml' 2026-05-11T15:53:33,052 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_aging.yaml' 2026-05-11T15:53:33,053 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_human_sexuality.yaml' 2026-05-11T15:53:33,054 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_international_law.yaml' 2026-05-11T15:53:33,056 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_jurisprudence.yaml' 2026-05-11T15:53:33,057 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_logical_fallacies.yaml' 2026-05-11T15:53:33,058 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_machine_learning.yaml' 2026-05-11T15:53:33,059 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_management.yaml' 2026-05-11T15:53:33,060 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_marketing.yaml' 2026-05-11T15:53:33,061 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_medical_genetics.yaml' 2026-05-11T15:53:33,063 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_miscellaneous.yaml' 2026-05-11T15:53:33,064 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_disputes.yaml' 2026-05-11T15:53:33,065 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_moral_scenarios.yaml' 2026-05-11T15:53:33,066 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_nutrition.yaml' 2026-05-11T15:53:33,068 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_philosophy.yaml' 2026-05-11T15:53:33,069 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_prehistory.yaml' 2026-05-11T15:53:33,070 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_accounting.yaml' 2026-05-11T15:53:33,071 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_law.yaml' 2026-05-11T15:53:33,072 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_medicine.yaml' 2026-05-11T15:53:33,073 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_professional_psychology.yaml' 2026-05-11T15:53:33,075 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_public_relations.yaml' 2026-05-11T15:53:33,076 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_security_studies.yaml' 2026-05-11T15:53:33,077 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_sociology.yaml' 2026-05-11T15:53:33,078 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_us_foreign_policy.yaml' 2026-05-11T15:53:33,079 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_virology.yaml' 2026-05-11T15:53:33,081 adding 'lm_eval/tasks/global_mmlu/full/fr/global_mmlu_full_fr_world_religions.yaml' 2026-05-11T15:53:33,082 adding 'lm_eval/tasks/global_mmlu/full/fr/utils.py' 2026-05-11T15:53:33,085 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha.yaml' 2026-05-11T15:53:33,086 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_humanities.yaml' 2026-05-11T15:53:33,087 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_other.yaml' 2026-05-11T15:53:33,088 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_social_sciences.yaml' 2026-05-11T15:53:33,090 adding 'lm_eval/tasks/global_mmlu/full/ha/_global_mmlu_full_ha_stem.yaml' 2026-05-11T15:53:33,091 adding 'lm_eval/tasks/global_mmlu/full/ha/_ha_template_yaml' 2026-05-11T15:53:33,092 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_abstract_algebra.yaml' 2026-05-11T15:53:33,093 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_anatomy.yaml' 2026-05-11T15:53:33,094 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_astronomy.yaml' 2026-05-11T15:53:33,096 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_business_ethics.yaml' 2026-05-11T15:53:33,097 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_clinical_knowledge.yaml' 2026-05-11T15:53:33,098 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_biology.yaml' 2026-05-11T15:53:33,099 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_chemistry.yaml' 2026-05-11T15:53:33,100 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_computer_science.yaml' 2026-05-11T15:53:33,101 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_mathematics.yaml' 2026-05-11T15:53:33,103 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_medicine.yaml' 2026-05-11T15:53:33,104 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_college_physics.yaml' 2026-05-11T15:53:33,105 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_computer_security.yaml' 2026-05-11T15:53:33,106 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_conceptual_physics.yaml' 2026-05-11T15:53:33,107 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_econometrics.yaml' 2026-05-11T15:53:33,109 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_electrical_engineering.yaml' 2026-05-11T15:53:33,110 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_elementary_mathematics.yaml' 2026-05-11T15:53:33,111 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_formal_logic.yaml' 2026-05-11T15:53:33,112 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_global_facts.yaml' 2026-05-11T15:53:33,113 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_biology.yaml' 2026-05-11T15:53:33,115 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_chemistry.yaml' 2026-05-11T15:53:33,116 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_computer_science.yaml' 2026-05-11T15:53:33,117 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_european_history.yaml' 2026-05-11T15:53:33,118 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_geography.yaml' 2026-05-11T15:53:33,119 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,120 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,121 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_mathematics.yaml' 2026-05-11T15:53:33,122 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_microeconomics.yaml' 2026-05-11T15:53:33,124 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_physics.yaml' 2026-05-11T15:53:33,125 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_psychology.yaml' 2026-05-11T15:53:33,126 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_statistics.yaml' 2026-05-11T15:53:33,127 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_us_history.yaml' 2026-05-11T15:53:33,128 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_high_school_world_history.yaml' 2026-05-11T15:53:33,130 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_aging.yaml' 2026-05-11T15:53:33,131 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_human_sexuality.yaml' 2026-05-11T15:53:33,132 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_international_law.yaml' 2026-05-11T15:53:33,133 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_jurisprudence.yaml' 2026-05-11T15:53:33,134 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_logical_fallacies.yaml' 2026-05-11T15:53:33,135 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_machine_learning.yaml' 2026-05-11T15:53:33,137 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_management.yaml' 2026-05-11T15:53:33,138 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_marketing.yaml' 2026-05-11T15:53:33,139 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_medical_genetics.yaml' 2026-05-11T15:53:33,140 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_miscellaneous.yaml' 2026-05-11T15:53:33,141 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_disputes.yaml' 2026-05-11T15:53:33,142 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_moral_scenarios.yaml' 2026-05-11T15:53:33,144 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_nutrition.yaml' 2026-05-11T15:53:33,145 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_philosophy.yaml' 2026-05-11T15:53:33,146 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_prehistory.yaml' 2026-05-11T15:53:33,147 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_accounting.yaml' 2026-05-11T15:53:33,149 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_law.yaml' 2026-05-11T15:53:33,150 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_medicine.yaml' 2026-05-11T15:53:33,151 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_professional_psychology.yaml' 2026-05-11T15:53:33,152 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_public_relations.yaml' 2026-05-11T15:53:33,153 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_security_studies.yaml' 2026-05-11T15:53:33,155 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_sociology.yaml' 2026-05-11T15:53:33,156 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_us_foreign_policy.yaml' 2026-05-11T15:53:33,157 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_virology.yaml' 2026-05-11T15:53:33,158 adding 'lm_eval/tasks/global_mmlu/full/ha/global_mmlu_full_ha_world_religions.yaml' 2026-05-11T15:53:33,159 adding 'lm_eval/tasks/global_mmlu/full/ha/utils.py' 2026-05-11T15:53:33,162 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he.yaml' 2026-05-11T15:53:33,164 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_humanities.yaml' 2026-05-11T15:53:33,165 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_other.yaml' 2026-05-11T15:53:33,166 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_social_sciences.yaml' 2026-05-11T15:53:33,167 adding 'lm_eval/tasks/global_mmlu/full/he/_global_mmlu_full_he_stem.yaml' 2026-05-11T15:53:33,168 adding 'lm_eval/tasks/global_mmlu/full/he/_he_template_yaml' 2026-05-11T15:53:33,169 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_abstract_algebra.yaml' 2026-05-11T15:53:33,170 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_anatomy.yaml' 2026-05-11T15:53:33,172 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_astronomy.yaml' 2026-05-11T15:53:33,173 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_business_ethics.yaml' 2026-05-11T15:53:33,174 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_clinical_knowledge.yaml' 2026-05-11T15:53:33,175 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_biology.yaml' 2026-05-11T15:53:33,176 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_chemistry.yaml' 2026-05-11T15:53:33,177 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_computer_science.yaml' 2026-05-11T15:53:33,179 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_mathematics.yaml' 2026-05-11T15:53:33,180 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_medicine.yaml' 2026-05-11T15:53:33,181 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_college_physics.yaml' 2026-05-11T15:53:33,182 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_computer_security.yaml' 2026-05-11T15:53:33,183 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_conceptual_physics.yaml' 2026-05-11T15:53:33,185 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_econometrics.yaml' 2026-05-11T15:53:33,186 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_electrical_engineering.yaml' 2026-05-11T15:53:33,187 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_elementary_mathematics.yaml' 2026-05-11T15:53:33,188 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_formal_logic.yaml' 2026-05-11T15:53:33,189 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_global_facts.yaml' 2026-05-11T15:53:33,190 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_biology.yaml' 2026-05-11T15:53:33,192 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_chemistry.yaml' 2026-05-11T15:53:33,193 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_computer_science.yaml' 2026-05-11T15:53:33,194 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_european_history.yaml' 2026-05-11T15:53:33,195 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_geography.yaml' 2026-05-11T15:53:33,196 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,198 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,199 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_mathematics.yaml' 2026-05-11T15:53:33,200 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_microeconomics.yaml' 2026-05-11T15:53:33,201 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_physics.yaml' 2026-05-11T15:53:33,202 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_psychology.yaml' 2026-05-11T15:53:33,204 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_statistics.yaml' 2026-05-11T15:53:33,205 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_us_history.yaml' 2026-05-11T15:53:33,206 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_high_school_world_history.yaml' 2026-05-11T15:53:33,207 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_aging.yaml' 2026-05-11T15:53:33,208 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_human_sexuality.yaml' 2026-05-11T15:53:33,209 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_international_law.yaml' 2026-05-11T15:53:33,210 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_jurisprudence.yaml' 2026-05-11T15:53:33,212 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_logical_fallacies.yaml' 2026-05-11T15:53:33,213 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_machine_learning.yaml' 2026-05-11T15:53:33,214 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_management.yaml' 2026-05-11T15:53:33,215 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_marketing.yaml' 2026-05-11T15:53:33,216 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_medical_genetics.yaml' 2026-05-11T15:53:33,218 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_miscellaneous.yaml' 2026-05-11T15:53:33,219 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_disputes.yaml' 2026-05-11T15:53:33,220 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_moral_scenarios.yaml' 2026-05-11T15:53:33,221 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_nutrition.yaml' 2026-05-11T15:53:33,222 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_philosophy.yaml' 2026-05-11T15:53:33,223 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_prehistory.yaml' 2026-05-11T15:53:33,225 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_accounting.yaml' 2026-05-11T15:53:33,226 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_law.yaml' 2026-05-11T15:53:33,227 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_medicine.yaml' 2026-05-11T15:53:33,228 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_professional_psychology.yaml' 2026-05-11T15:53:33,229 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_public_relations.yaml' 2026-05-11T15:53:33,231 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_security_studies.yaml' 2026-05-11T15:53:33,232 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_sociology.yaml' 2026-05-11T15:53:33,233 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_us_foreign_policy.yaml' 2026-05-11T15:53:33,234 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_virology.yaml' 2026-05-11T15:53:33,235 adding 'lm_eval/tasks/global_mmlu/full/he/global_mmlu_full_he_world_religions.yaml' 2026-05-11T15:53:33,237 adding 'lm_eval/tasks/global_mmlu/full/he/utils.py' 2026-05-11T15:53:33,239 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi.yaml' 2026-05-11T15:53:33,241 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_humanities.yaml' 2026-05-11T15:53:33,242 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_other.yaml' 2026-05-11T15:53:33,243 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_social_sciences.yaml' 2026-05-11T15:53:33,245 adding 'lm_eval/tasks/global_mmlu/full/hi/_global_mmlu_full_hi_stem.yaml' 2026-05-11T15:53:33,246 adding 'lm_eval/tasks/global_mmlu/full/hi/_hi_template_yaml' 2026-05-11T15:53:33,247 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_abstract_algebra.yaml' 2026-05-11T15:53:33,248 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_anatomy.yaml' 2026-05-11T15:53:33,249 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_astronomy.yaml' 2026-05-11T15:53:33,251 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_business_ethics.yaml' 2026-05-11T15:53:33,252 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_clinical_knowledge.yaml' 2026-05-11T15:53:33,253 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_biology.yaml' 2026-05-11T15:53:33,254 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_chemistry.yaml' 2026-05-11T15:53:33,255 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_computer_science.yaml' 2026-05-11T15:53:33,256 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_mathematics.yaml' 2026-05-11T15:53:33,257 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_medicine.yaml' 2026-05-11T15:53:33,258 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_college_physics.yaml' 2026-05-11T15:53:33,260 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_computer_security.yaml' 2026-05-11T15:53:33,261 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_conceptual_physics.yaml' 2026-05-11T15:53:33,262 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_econometrics.yaml' 2026-05-11T15:53:33,263 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_electrical_engineering.yaml' 2026-05-11T15:53:33,264 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_elementary_mathematics.yaml' 2026-05-11T15:53:33,265 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_formal_logic.yaml' 2026-05-11T15:53:33,266 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_global_facts.yaml' 2026-05-11T15:53:33,268 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_biology.yaml' 2026-05-11T15:53:33,269 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_chemistry.yaml' 2026-05-11T15:53:33,270 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_computer_science.yaml' 2026-05-11T15:53:33,271 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_european_history.yaml' 2026-05-11T15:53:33,273 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_geography.yaml' 2026-05-11T15:53:33,274 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,275 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,276 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_mathematics.yaml' 2026-05-11T15:53:33,277 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_microeconomics.yaml' 2026-05-11T15:53:33,279 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_physics.yaml' 2026-05-11T15:53:33,280 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_psychology.yaml' 2026-05-11T15:53:33,281 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_statistics.yaml' 2026-05-11T15:53:33,282 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_us_history.yaml' 2026-05-11T15:53:33,283 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_high_school_world_history.yaml' 2026-05-11T15:53:33,284 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_aging.yaml' 2026-05-11T15:53:33,286 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_human_sexuality.yaml' 2026-05-11T15:53:33,287 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_international_law.yaml' 2026-05-11T15:53:33,288 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_jurisprudence.yaml' 2026-05-11T15:53:33,289 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_logical_fallacies.yaml' 2026-05-11T15:53:33,290 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_machine_learning.yaml' 2026-05-11T15:53:33,292 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_management.yaml' 2026-05-11T15:53:33,293 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_marketing.yaml' 2026-05-11T15:53:33,294 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_medical_genetics.yaml' 2026-05-11T15:53:33,295 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_miscellaneous.yaml' 2026-05-11T15:53:33,296 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_disputes.yaml' 2026-05-11T15:53:33,297 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_moral_scenarios.yaml' 2026-05-11T15:53:33,298 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_nutrition.yaml' 2026-05-11T15:53:33,299 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_philosophy.yaml' 2026-05-11T15:53:33,300 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_prehistory.yaml' 2026-05-11T15:53:33,302 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_accounting.yaml' 2026-05-11T15:53:33,303 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_law.yaml' 2026-05-11T15:53:33,304 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_medicine.yaml' 2026-05-11T15:53:33,305 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_professional_psychology.yaml' 2026-05-11T15:53:33,306 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_public_relations.yaml' 2026-05-11T15:53:33,307 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_security_studies.yaml' 2026-05-11T15:53:33,309 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_sociology.yaml' 2026-05-11T15:53:33,310 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_us_foreign_policy.yaml' 2026-05-11T15:53:33,311 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_virology.yaml' 2026-05-11T15:53:33,312 adding 'lm_eval/tasks/global_mmlu/full/hi/global_mmlu_full_hi_world_religions.yaml' 2026-05-11T15:53:33,314 adding 'lm_eval/tasks/global_mmlu/full/hi/utils.py' 2026-05-11T15:53:33,316 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id.yaml' 2026-05-11T15:53:33,318 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_humanities.yaml' 2026-05-11T15:53:33,319 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_other.yaml' 2026-05-11T15:53:33,320 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_social_sciences.yaml' 2026-05-11T15:53:33,321 adding 'lm_eval/tasks/global_mmlu/full/id/_global_mmlu_full_id_stem.yaml' 2026-05-11T15:53:33,322 adding 'lm_eval/tasks/global_mmlu/full/id/_id_template_yaml' 2026-05-11T15:53:33,324 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_abstract_algebra.yaml' 2026-05-11T15:53:33,325 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_anatomy.yaml' 2026-05-11T15:53:33,326 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_astronomy.yaml' 2026-05-11T15:53:33,327 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_business_ethics.yaml' 2026-05-11T15:53:33,328 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_clinical_knowledge.yaml' 2026-05-11T15:53:33,330 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_biology.yaml' 2026-05-11T15:53:33,331 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_chemistry.yaml' 2026-05-11T15:53:33,332 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_computer_science.yaml' 2026-05-11T15:53:33,333 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_mathematics.yaml' 2026-05-11T15:53:33,334 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_medicine.yaml' 2026-05-11T15:53:33,335 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_college_physics.yaml' 2026-05-11T15:53:33,337 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_computer_security.yaml' 2026-05-11T15:53:33,338 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_conceptual_physics.yaml' 2026-05-11T15:53:33,339 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_econometrics.yaml' 2026-05-11T15:53:33,340 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_electrical_engineering.yaml' 2026-05-11T15:53:33,341 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_elementary_mathematics.yaml' 2026-05-11T15:53:33,343 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_formal_logic.yaml' 2026-05-11T15:53:33,344 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_global_facts.yaml' 2026-05-11T15:53:33,345 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_biology.yaml' 2026-05-11T15:53:33,346 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_chemistry.yaml' 2026-05-11T15:53:33,347 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_computer_science.yaml' 2026-05-11T15:53:33,348 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_european_history.yaml' 2026-05-11T15:53:33,350 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_geography.yaml' 2026-05-11T15:53:33,351 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,352 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,353 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_mathematics.yaml' 2026-05-11T15:53:33,354 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_microeconomics.yaml' 2026-05-11T15:53:33,355 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_physics.yaml' 2026-05-11T15:53:33,356 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_psychology.yaml' 2026-05-11T15:53:33,357 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_statistics.yaml' 2026-05-11T15:53:33,359 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_us_history.yaml' 2026-05-11T15:53:33,360 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_high_school_world_history.yaml' 2026-05-11T15:53:33,361 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_aging.yaml' 2026-05-11T15:53:33,362 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_human_sexuality.yaml' 2026-05-11T15:53:33,363 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_international_law.yaml' 2026-05-11T15:53:33,364 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_jurisprudence.yaml' 2026-05-11T15:53:33,366 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_logical_fallacies.yaml' 2026-05-11T15:53:33,367 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_machine_learning.yaml' 2026-05-11T15:53:33,368 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_management.yaml' 2026-05-11T15:53:33,369 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_marketing.yaml' 2026-05-11T15:53:33,370 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_medical_genetics.yaml' 2026-05-11T15:53:33,372 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_miscellaneous.yaml' 2026-05-11T15:53:33,373 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_disputes.yaml' 2026-05-11T15:53:33,374 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_moral_scenarios.yaml' 2026-05-11T15:53:33,375 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_nutrition.yaml' 2026-05-11T15:53:33,376 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_philosophy.yaml' 2026-05-11T15:53:33,378 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_prehistory.yaml' 2026-05-11T15:53:33,379 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_accounting.yaml' 2026-05-11T15:53:33,380 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_law.yaml' 2026-05-11T15:53:33,381 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_medicine.yaml' 2026-05-11T15:53:33,382 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_professional_psychology.yaml' 2026-05-11T15:53:33,383 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_public_relations.yaml' 2026-05-11T15:53:33,385 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_security_studies.yaml' 2026-05-11T15:53:33,386 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_sociology.yaml' 2026-05-11T15:53:33,387 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_us_foreign_policy.yaml' 2026-05-11T15:53:33,388 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_virology.yaml' 2026-05-11T15:53:33,389 adding 'lm_eval/tasks/global_mmlu/full/id/global_mmlu_full_id_world_religions.yaml' 2026-05-11T15:53:33,390 adding 'lm_eval/tasks/global_mmlu/full/id/utils.py' 2026-05-11T15:53:33,393 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig.yaml' 2026-05-11T15:53:33,394 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_humanities.yaml' 2026-05-11T15:53:33,395 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_other.yaml' 2026-05-11T15:53:33,397 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_social_sciences.yaml' 2026-05-11T15:53:33,398 adding 'lm_eval/tasks/global_mmlu/full/ig/_global_mmlu_full_ig_stem.yaml' 2026-05-11T15:53:33,399 adding 'lm_eval/tasks/global_mmlu/full/ig/_ig_template_yaml' 2026-05-11T15:53:33,400 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_abstract_algebra.yaml' 2026-05-11T15:53:33,401 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_anatomy.yaml' 2026-05-11T15:53:33,402 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_astronomy.yaml' 2026-05-11T15:53:33,403 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_business_ethics.yaml' 2026-05-11T15:53:33,405 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_clinical_knowledge.yaml' 2026-05-11T15:53:33,406 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_biology.yaml' 2026-05-11T15:53:33,407 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_chemistry.yaml' 2026-05-11T15:53:33,408 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_computer_science.yaml' 2026-05-11T15:53:33,409 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_mathematics.yaml' 2026-05-11T15:53:33,411 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_medicine.yaml' 2026-05-11T15:53:33,412 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_college_physics.yaml' 2026-05-11T15:53:33,413 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_computer_security.yaml' 2026-05-11T15:53:33,414 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_conceptual_physics.yaml' 2026-05-11T15:53:33,415 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_econometrics.yaml' 2026-05-11T15:53:33,417 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_electrical_engineering.yaml' 2026-05-11T15:53:33,418 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_elementary_mathematics.yaml' 2026-05-11T15:53:33,419 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_formal_logic.yaml' 2026-05-11T15:53:33,420 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_global_facts.yaml' 2026-05-11T15:53:33,421 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_biology.yaml' 2026-05-11T15:53:33,422 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_chemistry.yaml' 2026-05-11T15:53:33,424 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_computer_science.yaml' 2026-05-11T15:53:33,425 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_european_history.yaml' 2026-05-11T15:53:33,426 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_geography.yaml' 2026-05-11T15:53:33,427 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,428 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,429 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_mathematics.yaml' 2026-05-11T15:53:33,431 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_microeconomics.yaml' 2026-05-11T15:53:33,432 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_physics.yaml' 2026-05-11T15:53:33,433 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_psychology.yaml' 2026-05-11T15:53:33,434 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_statistics.yaml' 2026-05-11T15:53:33,435 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_us_history.yaml' 2026-05-11T15:53:33,436 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_high_school_world_history.yaml' 2026-05-11T15:53:33,437 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_aging.yaml' 2026-05-11T15:53:33,439 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_human_sexuality.yaml' 2026-05-11T15:53:33,440 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_international_law.yaml' 2026-05-11T15:53:33,442 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_jurisprudence.yaml' 2026-05-11T15:53:33,443 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_logical_fallacies.yaml' 2026-05-11T15:53:33,444 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_machine_learning.yaml' 2026-05-11T15:53:33,445 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_management.yaml' 2026-05-11T15:53:33,447 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_marketing.yaml' 2026-05-11T15:53:33,448 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_medical_genetics.yaml' 2026-05-11T15:53:33,450 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_miscellaneous.yaml' 2026-05-11T15:53:33,451 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_disputes.yaml' 2026-05-11T15:53:33,452 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_moral_scenarios.yaml' 2026-05-11T15:53:33,454 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_nutrition.yaml' 2026-05-11T15:53:33,455 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_philosophy.yaml' 2026-05-11T15:53:33,456 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_prehistory.yaml' 2026-05-11T15:53:33,457 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_accounting.yaml' 2026-05-11T15:53:33,459 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_law.yaml' 2026-05-11T15:53:33,460 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_medicine.yaml' 2026-05-11T15:53:33,461 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_professional_psychology.yaml' 2026-05-11T15:53:33,463 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_public_relations.yaml' 2026-05-11T15:53:33,464 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_security_studies.yaml' 2026-05-11T15:53:33,466 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_sociology.yaml' 2026-05-11T15:53:33,467 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_us_foreign_policy.yaml' 2026-05-11T15:53:33,468 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_virology.yaml' 2026-05-11T15:53:33,470 adding 'lm_eval/tasks/global_mmlu/full/ig/global_mmlu_full_ig_world_religions.yaml' 2026-05-11T15:53:33,471 adding 'lm_eval/tasks/global_mmlu/full/ig/utils.py' 2026-05-11T15:53:33,475 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it.yaml' 2026-05-11T15:53:33,476 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_humanities.yaml' 2026-05-11T15:53:33,478 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_other.yaml' 2026-05-11T15:53:33,479 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_social_sciences.yaml' 2026-05-11T15:53:33,480 adding 'lm_eval/tasks/global_mmlu/full/it/_global_mmlu_full_it_stem.yaml' 2026-05-11T15:53:33,482 adding 'lm_eval/tasks/global_mmlu/full/it/_it_template_yaml' 2026-05-11T15:53:33,483 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_abstract_algebra.yaml' 2026-05-11T15:53:33,484 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_anatomy.yaml' 2026-05-11T15:53:33,486 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_astronomy.yaml' 2026-05-11T15:53:33,487 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_business_ethics.yaml' 2026-05-11T15:53:33,489 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_clinical_knowledge.yaml' 2026-05-11T15:53:33,490 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_biology.yaml' 2026-05-11T15:53:33,491 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_chemistry.yaml' 2026-05-11T15:53:33,493 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_computer_science.yaml' 2026-05-11T15:53:33,494 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_mathematics.yaml' 2026-05-11T15:53:33,496 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_medicine.yaml' 2026-05-11T15:53:33,497 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_college_physics.yaml' 2026-05-11T15:53:33,498 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_computer_security.yaml' 2026-05-11T15:53:33,500 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_conceptual_physics.yaml' 2026-05-11T15:53:33,501 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_econometrics.yaml' 2026-05-11T15:53:33,502 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_electrical_engineering.yaml' 2026-05-11T15:53:33,503 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_elementary_mathematics.yaml' 2026-05-11T15:53:33,504 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_formal_logic.yaml' 2026-05-11T15:53:33,506 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_global_facts.yaml' 2026-05-11T15:53:33,507 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_biology.yaml' 2026-05-11T15:53:33,508 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_chemistry.yaml' 2026-05-11T15:53:33,509 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_computer_science.yaml' 2026-05-11T15:53:33,510 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_european_history.yaml' 2026-05-11T15:53:33,512 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_geography.yaml' 2026-05-11T15:53:33,513 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,514 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,515 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_mathematics.yaml' 2026-05-11T15:53:33,516 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_microeconomics.yaml' 2026-05-11T15:53:33,518 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_physics.yaml' 2026-05-11T15:53:33,519 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_psychology.yaml' 2026-05-11T15:53:33,520 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_statistics.yaml' 2026-05-11T15:53:33,521 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_us_history.yaml' 2026-05-11T15:53:33,523 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_high_school_world_history.yaml' 2026-05-11T15:53:33,524 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_aging.yaml' 2026-05-11T15:53:33,525 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_human_sexuality.yaml' 2026-05-11T15:53:33,526 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_international_law.yaml' 2026-05-11T15:53:33,527 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_jurisprudence.yaml' 2026-05-11T15:53:33,529 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_logical_fallacies.yaml' 2026-05-11T15:53:33,530 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_machine_learning.yaml' 2026-05-11T15:53:33,531 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_management.yaml' 2026-05-11T15:53:33,532 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_marketing.yaml' 2026-05-11T15:53:33,533 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_medical_genetics.yaml' 2026-05-11T15:53:33,534 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_miscellaneous.yaml' 2026-05-11T15:53:33,536 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_disputes.yaml' 2026-05-11T15:53:33,537 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_moral_scenarios.yaml' 2026-05-11T15:53:33,538 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_nutrition.yaml' 2026-05-11T15:53:33,539 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_philosophy.yaml' 2026-05-11T15:53:33,540 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_prehistory.yaml' 2026-05-11T15:53:33,542 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_accounting.yaml' 2026-05-11T15:53:33,543 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_law.yaml' 2026-05-11T15:53:33,544 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_medicine.yaml' 2026-05-11T15:53:33,545 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_professional_psychology.yaml' 2026-05-11T15:53:33,546 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_public_relations.yaml' 2026-05-11T15:53:33,547 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_security_studies.yaml' 2026-05-11T15:53:33,549 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_sociology.yaml' 2026-05-11T15:53:33,550 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_us_foreign_policy.yaml' 2026-05-11T15:53:33,551 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_virology.yaml' 2026-05-11T15:53:33,552 adding 'lm_eval/tasks/global_mmlu/full/it/global_mmlu_full_it_world_religions.yaml' 2026-05-11T15:53:33,554 adding 'lm_eval/tasks/global_mmlu/full/it/utils.py' 2026-05-11T15:53:33,557 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja.yaml' 2026-05-11T15:53:33,558 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_humanities.yaml' 2026-05-11T15:53:33,559 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_other.yaml' 2026-05-11T15:53:33,560 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_social_sciences.yaml' 2026-05-11T15:53:33,562 adding 'lm_eval/tasks/global_mmlu/full/ja/_global_mmlu_full_ja_stem.yaml' 2026-05-11T15:53:33,563 adding 'lm_eval/tasks/global_mmlu/full/ja/_ja_template_yaml' 2026-05-11T15:53:33,564 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_abstract_algebra.yaml' 2026-05-11T15:53:33,565 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_anatomy.yaml' 2026-05-11T15:53:33,567 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_astronomy.yaml' 2026-05-11T15:53:33,568 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_business_ethics.yaml' 2026-05-11T15:53:33,569 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_clinical_knowledge.yaml' 2026-05-11T15:53:33,570 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_biology.yaml' 2026-05-11T15:53:33,572 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_chemistry.yaml' 2026-05-11T15:53:33,573 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_computer_science.yaml' 2026-05-11T15:53:33,574 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_mathematics.yaml' 2026-05-11T15:53:33,575 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_medicine.yaml' 2026-05-11T15:53:33,576 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_college_physics.yaml' 2026-05-11T15:53:33,577 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_computer_security.yaml' 2026-05-11T15:53:33,578 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_conceptual_physics.yaml' 2026-05-11T15:53:33,579 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_econometrics.yaml' 2026-05-11T15:53:33,581 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_electrical_engineering.yaml' 2026-05-11T15:53:33,582 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_elementary_mathematics.yaml' 2026-05-11T15:53:33,583 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_formal_logic.yaml' 2026-05-11T15:53:33,584 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_global_facts.yaml' 2026-05-11T15:53:33,585 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_biology.yaml' 2026-05-11T15:53:33,587 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_chemistry.yaml' 2026-05-11T15:53:33,588 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_computer_science.yaml' 2026-05-11T15:53:33,589 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_european_history.yaml' 2026-05-11T15:53:33,590 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_geography.yaml' 2026-05-11T15:53:33,591 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,593 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,594 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_mathematics.yaml' 2026-05-11T15:53:33,595 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_microeconomics.yaml' 2026-05-11T15:53:33,596 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_physics.yaml' 2026-05-11T15:53:33,597 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_psychology.yaml' 2026-05-11T15:53:33,599 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_statistics.yaml' 2026-05-11T15:53:33,600 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_us_history.yaml' 2026-05-11T15:53:33,601 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_high_school_world_history.yaml' 2026-05-11T15:53:33,602 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_aging.yaml' 2026-05-11T15:53:33,603 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_human_sexuality.yaml' 2026-05-11T15:53:33,605 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_international_law.yaml' 2026-05-11T15:53:33,606 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_jurisprudence.yaml' 2026-05-11T15:53:33,607 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_logical_fallacies.yaml' 2026-05-11T15:53:33,608 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_machine_learning.yaml' 2026-05-11T15:53:33,609 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_management.yaml' 2026-05-11T15:53:33,611 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_marketing.yaml' 2026-05-11T15:53:33,612 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_medical_genetics.yaml' 2026-05-11T15:53:33,613 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_miscellaneous.yaml' 2026-05-11T15:53:33,614 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_disputes.yaml' 2026-05-11T15:53:33,615 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_moral_scenarios.yaml' 2026-05-11T15:53:33,616 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_nutrition.yaml' 2026-05-11T15:53:33,618 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_philosophy.yaml' 2026-05-11T15:53:33,619 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_prehistory.yaml' 2026-05-11T15:53:33,621 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_accounting.yaml' 2026-05-11T15:53:33,622 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_law.yaml' 2026-05-11T15:53:33,623 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_medicine.yaml' 2026-05-11T15:53:33,625 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_professional_psychology.yaml' 2026-05-11T15:53:33,626 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_public_relations.yaml' 2026-05-11T15:53:33,628 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_security_studies.yaml' 2026-05-11T15:53:33,629 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_sociology.yaml' 2026-05-11T15:53:33,630 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_us_foreign_policy.yaml' 2026-05-11T15:53:33,632 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_virology.yaml' 2026-05-11T15:53:33,633 adding 'lm_eval/tasks/global_mmlu/full/ja/global_mmlu_full_ja_world_religions.yaml' 2026-05-11T15:53:33,635 adding 'lm_eval/tasks/global_mmlu/full/ja/utils.py' 2026-05-11T15:53:33,638 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko.yaml' 2026-05-11T15:53:33,640 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_humanities.yaml' 2026-05-11T15:53:33,641 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_other.yaml' 2026-05-11T15:53:33,643 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_social_sciences.yaml' 2026-05-11T15:53:33,644 adding 'lm_eval/tasks/global_mmlu/full/ko/_global_mmlu_full_ko_stem.yaml' 2026-05-11T15:53:33,645 adding 'lm_eval/tasks/global_mmlu/full/ko/_ko_template_yaml' 2026-05-11T15:53:33,647 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_abstract_algebra.yaml' 2026-05-11T15:53:33,648 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_anatomy.yaml' 2026-05-11T15:53:33,650 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_astronomy.yaml' 2026-05-11T15:53:33,651 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_business_ethics.yaml' 2026-05-11T15:53:33,653 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_clinical_knowledge.yaml' 2026-05-11T15:53:33,654 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_biology.yaml' 2026-05-11T15:53:33,656 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_chemistry.yaml' 2026-05-11T15:53:33,657 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_computer_science.yaml' 2026-05-11T15:53:33,659 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_mathematics.yaml' 2026-05-11T15:53:33,660 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_medicine.yaml' 2026-05-11T15:53:33,661 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_college_physics.yaml' 2026-05-11T15:53:33,663 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_computer_security.yaml' 2026-05-11T15:53:33,664 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_conceptual_physics.yaml' 2026-05-11T15:53:33,666 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_econometrics.yaml' 2026-05-11T15:53:33,667 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_electrical_engineering.yaml' 2026-05-11T15:53:33,669 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_elementary_mathematics.yaml' 2026-05-11T15:53:33,670 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_formal_logic.yaml' 2026-05-11T15:53:33,672 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_global_facts.yaml' 2026-05-11T15:53:33,673 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_biology.yaml' 2026-05-11T15:53:33,675 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_chemistry.yaml' 2026-05-11T15:53:33,676 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_computer_science.yaml' 2026-05-11T15:53:33,678 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_european_history.yaml' 2026-05-11T15:53:33,679 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_geography.yaml' 2026-05-11T15:53:33,681 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,682 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,684 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_mathematics.yaml' 2026-05-11T15:53:33,685 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_microeconomics.yaml' 2026-05-11T15:53:33,686 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_physics.yaml' 2026-05-11T15:53:33,688 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_psychology.yaml' 2026-05-11T15:53:33,689 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_statistics.yaml' 2026-05-11T15:53:33,691 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_us_history.yaml' 2026-05-11T15:53:33,692 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_high_school_world_history.yaml' 2026-05-11T15:53:33,693 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_aging.yaml' 2026-05-11T15:53:33,695 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_human_sexuality.yaml' 2026-05-11T15:53:33,696 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_international_law.yaml' 2026-05-11T15:53:33,697 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_jurisprudence.yaml' 2026-05-11T15:53:33,699 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_logical_fallacies.yaml' 2026-05-11T15:53:33,700 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_machine_learning.yaml' 2026-05-11T15:53:33,702 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_management.yaml' 2026-05-11T15:53:33,703 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_marketing.yaml' 2026-05-11T15:53:33,704 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_medical_genetics.yaml' 2026-05-11T15:53:33,706 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_miscellaneous.yaml' 2026-05-11T15:53:33,707 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_disputes.yaml' 2026-05-11T15:53:33,709 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_moral_scenarios.yaml' 2026-05-11T15:53:33,711 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_nutrition.yaml' 2026-05-11T15:53:33,712 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_philosophy.yaml' 2026-05-11T15:53:33,713 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_prehistory.yaml' 2026-05-11T15:53:33,715 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_accounting.yaml' 2026-05-11T15:53:33,716 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_law.yaml' 2026-05-11T15:53:33,718 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_medicine.yaml' 2026-05-11T15:53:33,719 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_professional_psychology.yaml' 2026-05-11T15:53:33,721 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_public_relations.yaml' 2026-05-11T15:53:33,722 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_security_studies.yaml' 2026-05-11T15:53:33,723 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_sociology.yaml' 2026-05-11T15:53:33,725 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_us_foreign_policy.yaml' 2026-05-11T15:53:33,726 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_virology.yaml' 2026-05-11T15:53:33,728 adding 'lm_eval/tasks/global_mmlu/full/ko/global_mmlu_full_ko_world_religions.yaml' 2026-05-11T15:53:33,729 adding 'lm_eval/tasks/global_mmlu/full/ko/utils.py' 2026-05-11T15:53:33,733 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky.yaml' 2026-05-11T15:53:33,734 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_humanities.yaml' 2026-05-11T15:53:33,735 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_other.yaml' 2026-05-11T15:53:33,737 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_social_sciences.yaml' 2026-05-11T15:53:33,738 adding 'lm_eval/tasks/global_mmlu/full/ky/_global_mmlu_full_ky_stem.yaml' 2026-05-11T15:53:33,739 adding 'lm_eval/tasks/global_mmlu/full/ky/_ky_template_yaml' 2026-05-11T15:53:33,740 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_abstract_algebra.yaml' 2026-05-11T15:53:33,742 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_anatomy.yaml' 2026-05-11T15:53:33,743 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_astronomy.yaml' 2026-05-11T15:53:33,744 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_business_ethics.yaml' 2026-05-11T15:53:33,745 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_clinical_knowledge.yaml' 2026-05-11T15:53:33,746 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_biology.yaml' 2026-05-11T15:53:33,747 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_chemistry.yaml' 2026-05-11T15:53:33,748 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_computer_science.yaml' 2026-05-11T15:53:33,750 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_mathematics.yaml' 2026-05-11T15:53:33,751 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_medicine.yaml' 2026-05-11T15:53:33,752 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_college_physics.yaml' 2026-05-11T15:53:33,753 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_computer_security.yaml' 2026-05-11T15:53:33,754 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_conceptual_physics.yaml' 2026-05-11T15:53:33,755 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_econometrics.yaml' 2026-05-11T15:53:33,757 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_electrical_engineering.yaml' 2026-05-11T15:53:33,758 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_elementary_mathematics.yaml' 2026-05-11T15:53:33,759 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_formal_logic.yaml' 2026-05-11T15:53:33,760 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_global_facts.yaml' 2026-05-11T15:53:33,761 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_biology.yaml' 2026-05-11T15:53:33,763 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_chemistry.yaml' 2026-05-11T15:53:33,764 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_computer_science.yaml' 2026-05-11T15:53:33,765 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_european_history.yaml' 2026-05-11T15:53:33,766 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_geography.yaml' 2026-05-11T15:53:33,767 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,768 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,770 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_mathematics.yaml' 2026-05-11T15:53:33,771 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_microeconomics.yaml' 2026-05-11T15:53:33,772 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_physics.yaml' 2026-05-11T15:53:33,773 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_psychology.yaml' 2026-05-11T15:53:33,774 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_statistics.yaml' 2026-05-11T15:53:33,775 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_us_history.yaml' 2026-05-11T15:53:33,776 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_high_school_world_history.yaml' 2026-05-11T15:53:33,777 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_aging.yaml' 2026-05-11T15:53:33,779 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_human_sexuality.yaml' 2026-05-11T15:53:33,780 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_international_law.yaml' 2026-05-11T15:53:33,781 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_jurisprudence.yaml' 2026-05-11T15:53:33,782 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_logical_fallacies.yaml' 2026-05-11T15:53:33,784 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_machine_learning.yaml' 2026-05-11T15:53:33,785 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_management.yaml' 2026-05-11T15:53:33,786 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_marketing.yaml' 2026-05-11T15:53:33,787 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_medical_genetics.yaml' 2026-05-11T15:53:33,788 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_miscellaneous.yaml' 2026-05-11T15:53:33,789 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_disputes.yaml' 2026-05-11T15:53:33,791 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_moral_scenarios.yaml' 2026-05-11T15:53:33,792 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_nutrition.yaml' 2026-05-11T15:53:33,793 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_philosophy.yaml' 2026-05-11T15:53:33,794 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_prehistory.yaml' 2026-05-11T15:53:33,795 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_accounting.yaml' 2026-05-11T15:53:33,796 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_law.yaml' 2026-05-11T15:53:33,797 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_medicine.yaml' 2026-05-11T15:53:33,799 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_professional_psychology.yaml' 2026-05-11T15:53:33,800 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_public_relations.yaml' 2026-05-11T15:53:33,801 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_security_studies.yaml' 2026-05-11T15:53:33,802 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_sociology.yaml' 2026-05-11T15:53:33,803 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_us_foreign_policy.yaml' 2026-05-11T15:53:33,805 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_virology.yaml' 2026-05-11T15:53:33,806 adding 'lm_eval/tasks/global_mmlu/full/ky/global_mmlu_full_ky_world_religions.yaml' 2026-05-11T15:53:33,807 adding 'lm_eval/tasks/global_mmlu/full/ky/utils.py' 2026-05-11T15:53:33,810 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt.yaml' 2026-05-11T15:53:33,812 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_humanities.yaml' 2026-05-11T15:53:33,813 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_other.yaml' 2026-05-11T15:53:33,814 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_social_sciences.yaml' 2026-05-11T15:53:33,815 adding 'lm_eval/tasks/global_mmlu/full/lt/_global_mmlu_full_lt_stem.yaml' 2026-05-11T15:53:33,816 adding 'lm_eval/tasks/global_mmlu/full/lt/_lt_template_yaml' 2026-05-11T15:53:33,817 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_abstract_algebra.yaml' 2026-05-11T15:53:33,818 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_anatomy.yaml' 2026-05-11T15:53:33,820 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_astronomy.yaml' 2026-05-11T15:53:33,821 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_business_ethics.yaml' 2026-05-11T15:53:33,822 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_clinical_knowledge.yaml' 2026-05-11T15:53:33,823 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_biology.yaml' 2026-05-11T15:53:33,824 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_chemistry.yaml' 2026-05-11T15:53:33,825 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_computer_science.yaml' 2026-05-11T15:53:33,828 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_mathematics.yaml' 2026-05-11T15:53:33,829 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_medicine.yaml' 2026-05-11T15:53:33,831 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_college_physics.yaml' 2026-05-11T15:53:33,832 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_computer_security.yaml' 2026-05-11T15:53:33,833 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_conceptual_physics.yaml' 2026-05-11T15:53:33,834 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_econometrics.yaml' 2026-05-11T15:53:33,835 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_electrical_engineering.yaml' 2026-05-11T15:53:33,836 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_elementary_mathematics.yaml' 2026-05-11T15:53:33,838 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_formal_logic.yaml' 2026-05-11T15:53:33,839 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_global_facts.yaml' 2026-05-11T15:53:33,840 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_biology.yaml' 2026-05-11T15:53:33,841 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_chemistry.yaml' 2026-05-11T15:53:33,842 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_computer_science.yaml' 2026-05-11T15:53:33,844 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_european_history.yaml' 2026-05-11T15:53:33,845 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_geography.yaml' 2026-05-11T15:53:33,846 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,847 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,848 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_mathematics.yaml' 2026-05-11T15:53:33,850 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_microeconomics.yaml' 2026-05-11T15:53:33,851 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_physics.yaml' 2026-05-11T15:53:33,852 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_psychology.yaml' 2026-05-11T15:53:33,853 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_statistics.yaml' 2026-05-11T15:53:33,854 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_us_history.yaml' 2026-05-11T15:53:33,856 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_high_school_world_history.yaml' 2026-05-11T15:53:33,857 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_aging.yaml' 2026-05-11T15:53:33,859 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_human_sexuality.yaml' 2026-05-11T15:53:33,860 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_international_law.yaml' 2026-05-11T15:53:33,861 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_jurisprudence.yaml' 2026-05-11T15:53:33,862 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_logical_fallacies.yaml' 2026-05-11T15:53:33,863 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_machine_learning.yaml' 2026-05-11T15:53:33,865 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_management.yaml' 2026-05-11T15:53:33,866 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_marketing.yaml' 2026-05-11T15:53:33,867 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_medical_genetics.yaml' 2026-05-11T15:53:33,868 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_miscellaneous.yaml' 2026-05-11T15:53:33,869 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_disputes.yaml' 2026-05-11T15:53:33,870 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_moral_scenarios.yaml' 2026-05-11T15:53:33,871 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_nutrition.yaml' 2026-05-11T15:53:33,873 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_philosophy.yaml' 2026-05-11T15:53:33,874 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_prehistory.yaml' 2026-05-11T15:53:33,875 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_accounting.yaml' 2026-05-11T15:53:33,876 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_law.yaml' 2026-05-11T15:53:33,877 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_medicine.yaml' 2026-05-11T15:53:33,878 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_professional_psychology.yaml' 2026-05-11T15:53:33,880 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_public_relations.yaml' 2026-05-11T15:53:33,881 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_security_studies.yaml' 2026-05-11T15:53:33,882 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_sociology.yaml' 2026-05-11T15:53:33,883 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_us_foreign_policy.yaml' 2026-05-11T15:53:33,884 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_virology.yaml' 2026-05-11T15:53:33,885 adding 'lm_eval/tasks/global_mmlu/full/lt/global_mmlu_full_lt_world_religions.yaml' 2026-05-11T15:53:33,887 adding 'lm_eval/tasks/global_mmlu/full/lt/utils.py' 2026-05-11T15:53:33,890 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg.yaml' 2026-05-11T15:53:33,891 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_humanities.yaml' 2026-05-11T15:53:33,892 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_other.yaml' 2026-05-11T15:53:33,893 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_social_sciences.yaml' 2026-05-11T15:53:33,894 adding 'lm_eval/tasks/global_mmlu/full/mg/_global_mmlu_full_mg_stem.yaml' 2026-05-11T15:53:33,896 adding 'lm_eval/tasks/global_mmlu/full/mg/_mg_template_yaml' 2026-05-11T15:53:33,897 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_abstract_algebra.yaml' 2026-05-11T15:53:33,898 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_anatomy.yaml' 2026-05-11T15:53:33,899 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_astronomy.yaml' 2026-05-11T15:53:33,901 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_business_ethics.yaml' 2026-05-11T15:53:33,902 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_clinical_knowledge.yaml' 2026-05-11T15:53:33,903 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_biology.yaml' 2026-05-11T15:53:33,904 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_chemistry.yaml' 2026-05-11T15:53:33,905 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_computer_science.yaml' 2026-05-11T15:53:33,907 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_mathematics.yaml' 2026-05-11T15:53:33,908 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_medicine.yaml' 2026-05-11T15:53:33,909 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_college_physics.yaml' 2026-05-11T15:53:33,910 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_computer_security.yaml' 2026-05-11T15:53:33,911 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_conceptual_physics.yaml' 2026-05-11T15:53:33,912 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_econometrics.yaml' 2026-05-11T15:53:33,913 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_electrical_engineering.yaml' 2026-05-11T15:53:33,915 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_elementary_mathematics.yaml' 2026-05-11T15:53:33,916 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_formal_logic.yaml' 2026-05-11T15:53:33,917 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_global_facts.yaml' 2026-05-11T15:53:33,918 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_biology.yaml' 2026-05-11T15:53:33,919 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_chemistry.yaml' 2026-05-11T15:53:33,920 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_computer_science.yaml' 2026-05-11T15:53:33,922 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_european_history.yaml' 2026-05-11T15:53:33,923 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_geography.yaml' 2026-05-11T15:53:33,924 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_government_and_politics.yaml' 2026-05-11T15:53:33,925 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_macroeconomics.yaml' 2026-05-11T15:53:33,926 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_mathematics.yaml' 2026-05-11T15:53:33,927 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_microeconomics.yaml' 2026-05-11T15:53:33,929 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_physics.yaml' 2026-05-11T15:53:33,930 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_psychology.yaml' 2026-05-11T15:53:33,931 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_statistics.yaml' 2026-05-11T15:53:33,932 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_us_history.yaml' 2026-05-11T15:53:33,933 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_high_school_world_history.yaml' 2026-05-11T15:53:33,934 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_aging.yaml' 2026-05-11T15:53:33,936 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_human_sexuality.yaml' 2026-05-11T15:53:33,937 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_international_law.yaml' 2026-05-11T15:53:33,938 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_jurisprudence.yaml' 2026-05-11T15:53:33,939 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_logical_fallacies.yaml' 2026-05-11T15:53:33,940 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_machine_learning.yaml' 2026-05-11T15:53:33,941 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_management.yaml' 2026-05-11T15:53:33,943 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_marketing.yaml' 2026-05-11T15:53:33,944 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_medical_genetics.yaml' 2026-05-11T15:53:33,945 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_miscellaneous.yaml' 2026-05-11T15:53:33,946 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_disputes.yaml' 2026-05-11T15:53:33,947 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_moral_scenarios.yaml' 2026-05-11T15:53:33,949 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_nutrition.yaml' 2026-05-11T15:53:33,950 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_philosophy.yaml' 2026-05-11T15:53:33,951 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_prehistory.yaml' 2026-05-11T15:53:33,952 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_accounting.yaml' 2026-05-11T15:53:33,953 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_law.yaml' 2026-05-11T15:53:33,954 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_medicine.yaml' 2026-05-11T15:53:33,956 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_professional_psychology.yaml' 2026-05-11T15:53:33,957 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_public_relations.yaml' 2026-05-11T15:53:33,958 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_security_studies.yaml' 2026-05-11T15:53:33,959 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_sociology.yaml' 2026-05-11T15:53:33,960 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_us_foreign_policy.yaml' 2026-05-11T15:53:33,961 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_virology.yaml' 2026-05-11T15:53:33,963 adding 'lm_eval/tasks/global_mmlu/full/mg/global_mmlu_full_mg_world_religions.yaml' 2026-05-11T15:53:33,964 adding 'lm_eval/tasks/global_mmlu/full/mg/utils.py' 2026-05-11T15:53:33,967 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms.yaml' 2026-05-11T15:53:33,968 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_humanities.yaml' 2026-05-11T15:53:33,969 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_other.yaml' 2026-05-11T15:53:33,970 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_social_sciences.yaml' 2026-05-11T15:53:33,972 adding 'lm_eval/tasks/global_mmlu/full/ms/_global_mmlu_full_ms_stem.yaml' 2026-05-11T15:53:33,973 adding 'lm_eval/tasks/global_mmlu/full/ms/_ms_template_yaml' 2026-05-11T15:53:33,974 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_abstract_algebra.yaml' 2026-05-11T15:53:33,975 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_anatomy.yaml' 2026-05-11T15:53:33,976 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_astronomy.yaml' 2026-05-11T15:53:33,978 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_business_ethics.yaml' 2026-05-11T15:53:33,979 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_clinical_knowledge.yaml' 2026-05-11T15:53:33,980 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_biology.yaml' 2026-05-11T15:53:33,981 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_chemistry.yaml' 2026-05-11T15:53:33,982 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_computer_science.yaml' 2026-05-11T15:53:33,984 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_mathematics.yaml' 2026-05-11T15:53:33,985 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_medicine.yaml' 2026-05-11T15:53:33,986 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_college_physics.yaml' 2026-05-11T15:53:33,987 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_computer_security.yaml' 2026-05-11T15:53:33,988 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_conceptual_physics.yaml' 2026-05-11T15:53:33,989 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_econometrics.yaml' 2026-05-11T15:53:33,991 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_electrical_engineering.yaml' 2026-05-11T15:53:33,992 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_elementary_mathematics.yaml' 2026-05-11T15:53:33,993 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_formal_logic.yaml' 2026-05-11T15:53:33,994 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_global_facts.yaml' 2026-05-11T15:53:33,995 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_biology.yaml' 2026-05-11T15:53:33,997 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_chemistry.yaml' 2026-05-11T15:53:33,998 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_computer_science.yaml' 2026-05-11T15:53:33,999 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_european_history.yaml' 2026-05-11T15:53:34,000 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_geography.yaml' 2026-05-11T15:53:34,001 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,002 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,003 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_mathematics.yaml' 2026-05-11T15:53:34,005 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_microeconomics.yaml' 2026-05-11T15:53:34,006 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_physics.yaml' 2026-05-11T15:53:34,007 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_psychology.yaml' 2026-05-11T15:53:34,008 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_statistics.yaml' 2026-05-11T15:53:34,009 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_us_history.yaml' 2026-05-11T15:53:34,011 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_high_school_world_history.yaml' 2026-05-11T15:53:34,012 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_aging.yaml' 2026-05-11T15:53:34,013 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_human_sexuality.yaml' 2026-05-11T15:53:34,014 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_international_law.yaml' 2026-05-11T15:53:34,015 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_jurisprudence.yaml' 2026-05-11T15:53:34,016 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_logical_fallacies.yaml' 2026-05-11T15:53:34,018 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_machine_learning.yaml' 2026-05-11T15:53:34,019 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_management.yaml' 2026-05-11T15:53:34,020 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_marketing.yaml' 2026-05-11T15:53:34,021 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_medical_genetics.yaml' 2026-05-11T15:53:34,022 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_miscellaneous.yaml' 2026-05-11T15:53:34,023 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_disputes.yaml' 2026-05-11T15:53:34,025 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_moral_scenarios.yaml' 2026-05-11T15:53:34,026 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_nutrition.yaml' 2026-05-11T15:53:34,027 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_philosophy.yaml' 2026-05-11T15:53:34,028 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_prehistory.yaml' 2026-05-11T15:53:34,029 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_accounting.yaml' 2026-05-11T15:53:34,031 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_law.yaml' 2026-05-11T15:53:34,032 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_medicine.yaml' 2026-05-11T15:53:34,033 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_professional_psychology.yaml' 2026-05-11T15:53:34,034 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_public_relations.yaml' 2026-05-11T15:53:34,035 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_security_studies.yaml' 2026-05-11T15:53:34,036 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_sociology.yaml' 2026-05-11T15:53:34,038 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_us_foreign_policy.yaml' 2026-05-11T15:53:34,039 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_virology.yaml' 2026-05-11T15:53:34,040 adding 'lm_eval/tasks/global_mmlu/full/ms/global_mmlu_full_ms_world_religions.yaml' 2026-05-11T15:53:34,041 adding 'lm_eval/tasks/global_mmlu/full/ms/utils.py' 2026-05-11T15:53:34,044 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne.yaml' 2026-05-11T15:53:34,045 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_humanities.yaml' 2026-05-11T15:53:34,046 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_other.yaml' 2026-05-11T15:53:34,047 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_social_sciences.yaml' 2026-05-11T15:53:34,049 adding 'lm_eval/tasks/global_mmlu/full/ne/_global_mmlu_full_ne_stem.yaml' 2026-05-11T15:53:34,050 adding 'lm_eval/tasks/global_mmlu/full/ne/_ne_template_yaml' 2026-05-11T15:53:34,051 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_abstract_algebra.yaml' 2026-05-11T15:53:34,052 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_anatomy.yaml' 2026-05-11T15:53:34,053 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_astronomy.yaml' 2026-05-11T15:53:34,054 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_business_ethics.yaml' 2026-05-11T15:53:34,055 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_clinical_knowledge.yaml' 2026-05-11T15:53:34,057 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_biology.yaml' 2026-05-11T15:53:34,058 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_chemistry.yaml' 2026-05-11T15:53:34,059 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_computer_science.yaml' 2026-05-11T15:53:34,060 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_mathematics.yaml' 2026-05-11T15:53:34,062 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_medicine.yaml' 2026-05-11T15:53:34,063 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_college_physics.yaml' 2026-05-11T15:53:34,064 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_computer_security.yaml' 2026-05-11T15:53:34,065 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_conceptual_physics.yaml' 2026-05-11T15:53:34,066 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_econometrics.yaml' 2026-05-11T15:53:34,068 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_electrical_engineering.yaml' 2026-05-11T15:53:34,069 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_elementary_mathematics.yaml' 2026-05-11T15:53:34,070 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_formal_logic.yaml' 2026-05-11T15:53:34,071 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_global_facts.yaml' 2026-05-11T15:53:34,072 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_biology.yaml' 2026-05-11T15:53:34,073 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_chemistry.yaml' 2026-05-11T15:53:34,075 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_computer_science.yaml' 2026-05-11T15:53:34,076 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_european_history.yaml' 2026-05-11T15:53:34,077 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_geography.yaml' 2026-05-11T15:53:34,078 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,079 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,081 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_mathematics.yaml' 2026-05-11T15:53:34,082 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_microeconomics.yaml' 2026-05-11T15:53:34,083 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_physics.yaml' 2026-05-11T15:53:34,084 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_psychology.yaml' 2026-05-11T15:53:34,085 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_statistics.yaml' 2026-05-11T15:53:34,086 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_us_history.yaml' 2026-05-11T15:53:34,087 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_high_school_world_history.yaml' 2026-05-11T15:53:34,089 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_aging.yaml' 2026-05-11T15:53:34,090 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_human_sexuality.yaml' 2026-05-11T15:53:34,091 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_international_law.yaml' 2026-05-11T15:53:34,092 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_jurisprudence.yaml' 2026-05-11T15:53:34,093 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_logical_fallacies.yaml' 2026-05-11T15:53:34,094 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_machine_learning.yaml' 2026-05-11T15:53:34,095 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_management.yaml' 2026-05-11T15:53:34,097 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_marketing.yaml' 2026-05-11T15:53:34,098 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_medical_genetics.yaml' 2026-05-11T15:53:34,099 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_miscellaneous.yaml' 2026-05-11T15:53:34,100 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_disputes.yaml' 2026-05-11T15:53:34,101 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_moral_scenarios.yaml' 2026-05-11T15:53:34,103 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_nutrition.yaml' 2026-05-11T15:53:34,104 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_philosophy.yaml' 2026-05-11T15:53:34,105 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_prehistory.yaml' 2026-05-11T15:53:34,106 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_accounting.yaml' 2026-05-11T15:53:34,108 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_law.yaml' 2026-05-11T15:53:34,109 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_medicine.yaml' 2026-05-11T15:53:34,110 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_professional_psychology.yaml' 2026-05-11T15:53:34,111 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_public_relations.yaml' 2026-05-11T15:53:34,112 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_security_studies.yaml' 2026-05-11T15:53:34,114 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_sociology.yaml' 2026-05-11T15:53:34,115 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_us_foreign_policy.yaml' 2026-05-11T15:53:34,116 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_virology.yaml' 2026-05-11T15:53:34,117 adding 'lm_eval/tasks/global_mmlu/full/ne/global_mmlu_full_ne_world_religions.yaml' 2026-05-11T15:53:34,119 adding 'lm_eval/tasks/global_mmlu/full/ne/utils.py' 2026-05-11T15:53:34,122 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl.yaml' 2026-05-11T15:53:34,123 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_humanities.yaml' 2026-05-11T15:53:34,124 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_other.yaml' 2026-05-11T15:53:34,125 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_social_sciences.yaml' 2026-05-11T15:53:34,126 adding 'lm_eval/tasks/global_mmlu/full/nl/_global_mmlu_full_nl_stem.yaml' 2026-05-11T15:53:34,128 adding 'lm_eval/tasks/global_mmlu/full/nl/_nl_template_yaml' 2026-05-11T15:53:34,129 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_abstract_algebra.yaml' 2026-05-11T15:53:34,130 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_anatomy.yaml' 2026-05-11T15:53:34,131 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_astronomy.yaml' 2026-05-11T15:53:34,132 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_business_ethics.yaml' 2026-05-11T15:53:34,134 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_clinical_knowledge.yaml' 2026-05-11T15:53:34,135 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_biology.yaml' 2026-05-11T15:53:34,136 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_chemistry.yaml' 2026-05-11T15:53:34,137 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_computer_science.yaml' 2026-05-11T15:53:34,138 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_mathematics.yaml' 2026-05-11T15:53:34,139 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_medicine.yaml' 2026-05-11T15:53:34,140 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_college_physics.yaml' 2026-05-11T15:53:34,141 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_computer_security.yaml' 2026-05-11T15:53:34,143 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_conceptual_physics.yaml' 2026-05-11T15:53:34,144 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_econometrics.yaml' 2026-05-11T15:53:34,145 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_electrical_engineering.yaml' 2026-05-11T15:53:34,146 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_elementary_mathematics.yaml' 2026-05-11T15:53:34,147 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_formal_logic.yaml' 2026-05-11T15:53:34,148 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_global_facts.yaml' 2026-05-11T15:53:34,150 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_biology.yaml' 2026-05-11T15:53:34,151 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_chemistry.yaml' 2026-05-11T15:53:34,152 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_computer_science.yaml' 2026-05-11T15:53:34,153 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_european_history.yaml' 2026-05-11T15:53:34,154 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_geography.yaml' 2026-05-11T15:53:34,156 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,157 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,158 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_mathematics.yaml' 2026-05-11T15:53:34,159 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_microeconomics.yaml' 2026-05-11T15:53:34,160 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_physics.yaml' 2026-05-11T15:53:34,162 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_psychology.yaml' 2026-05-11T15:53:34,163 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_statistics.yaml' 2026-05-11T15:53:34,164 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_us_history.yaml' 2026-05-11T15:53:34,165 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_high_school_world_history.yaml' 2026-05-11T15:53:34,166 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_aging.yaml' 2026-05-11T15:53:34,168 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_human_sexuality.yaml' 2026-05-11T15:53:34,169 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_international_law.yaml' 2026-05-11T15:53:34,170 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_jurisprudence.yaml' 2026-05-11T15:53:34,171 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_logical_fallacies.yaml' 2026-05-11T15:53:34,172 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_machine_learning.yaml' 2026-05-11T15:53:34,174 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_management.yaml' 2026-05-11T15:53:34,175 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_marketing.yaml' 2026-05-11T15:53:34,176 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_medical_genetics.yaml' 2026-05-11T15:53:34,177 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_miscellaneous.yaml' 2026-05-11T15:53:34,178 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_disputes.yaml' 2026-05-11T15:53:34,179 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_moral_scenarios.yaml' 2026-05-11T15:53:34,180 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_nutrition.yaml' 2026-05-11T15:53:34,181 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_philosophy.yaml' 2026-05-11T15:53:34,183 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_prehistory.yaml' 2026-05-11T15:53:34,184 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_accounting.yaml' 2026-05-11T15:53:34,185 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_law.yaml' 2026-05-11T15:53:34,186 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_medicine.yaml' 2026-05-11T15:53:34,187 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_professional_psychology.yaml' 2026-05-11T15:53:34,188 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_public_relations.yaml' 2026-05-11T15:53:34,190 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_security_studies.yaml' 2026-05-11T15:53:34,191 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_sociology.yaml' 2026-05-11T15:53:34,192 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_us_foreign_policy.yaml' 2026-05-11T15:53:34,193 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_virology.yaml' 2026-05-11T15:53:34,194 adding 'lm_eval/tasks/global_mmlu/full/nl/global_mmlu_full_nl_world_religions.yaml' 2026-05-11T15:53:34,196 adding 'lm_eval/tasks/global_mmlu/full/nl/utils.py' 2026-05-11T15:53:34,199 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny.yaml' 2026-05-11T15:53:34,200 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_humanities.yaml' 2026-05-11T15:53:34,201 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_other.yaml' 2026-05-11T15:53:34,202 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_social_sciences.yaml' 2026-05-11T15:53:34,204 adding 'lm_eval/tasks/global_mmlu/full/ny/_global_mmlu_full_ny_stem.yaml' 2026-05-11T15:53:34,205 adding 'lm_eval/tasks/global_mmlu/full/ny/_ny_template_yaml' 2026-05-11T15:53:34,206 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_abstract_algebra.yaml' 2026-05-11T15:53:34,207 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_anatomy.yaml' 2026-05-11T15:53:34,209 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_astronomy.yaml' 2026-05-11T15:53:34,210 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_business_ethics.yaml' 2026-05-11T15:53:34,211 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_clinical_knowledge.yaml' 2026-05-11T15:53:34,212 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_biology.yaml' 2026-05-11T15:53:34,213 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_chemistry.yaml' 2026-05-11T15:53:34,215 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_computer_science.yaml' 2026-05-11T15:53:34,216 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_mathematics.yaml' 2026-05-11T15:53:34,217 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_medicine.yaml' 2026-05-11T15:53:34,218 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_college_physics.yaml' 2026-05-11T15:53:34,219 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_computer_security.yaml' 2026-05-11T15:53:34,221 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_conceptual_physics.yaml' 2026-05-11T15:53:34,222 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_econometrics.yaml' 2026-05-11T15:53:34,223 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_electrical_engineering.yaml' 2026-05-11T15:53:34,224 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_elementary_mathematics.yaml' 2026-05-11T15:53:34,225 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_formal_logic.yaml' 2026-05-11T15:53:34,226 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_global_facts.yaml' 2026-05-11T15:53:34,227 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_biology.yaml' 2026-05-11T15:53:34,228 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_chemistry.yaml' 2026-05-11T15:53:34,229 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_computer_science.yaml' 2026-05-11T15:53:34,231 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_european_history.yaml' 2026-05-11T15:53:34,232 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_geography.yaml' 2026-05-11T15:53:34,233 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,234 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,235 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_mathematics.yaml' 2026-05-11T15:53:34,236 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_microeconomics.yaml' 2026-05-11T15:53:34,237 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_physics.yaml' 2026-05-11T15:53:34,239 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_psychology.yaml' 2026-05-11T15:53:34,240 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_statistics.yaml' 2026-05-11T15:53:34,241 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_us_history.yaml' 2026-05-11T15:53:34,242 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_high_school_world_history.yaml' 2026-05-11T15:53:34,243 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_aging.yaml' 2026-05-11T15:53:34,245 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_human_sexuality.yaml' 2026-05-11T15:53:34,246 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_international_law.yaml' 2026-05-11T15:53:34,247 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_jurisprudence.yaml' 2026-05-11T15:53:34,248 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_logical_fallacies.yaml' 2026-05-11T15:53:34,249 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_machine_learning.yaml' 2026-05-11T15:53:34,251 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_management.yaml' 2026-05-11T15:53:34,252 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_marketing.yaml' 2026-05-11T15:53:34,253 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_medical_genetics.yaml' 2026-05-11T15:53:34,254 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_miscellaneous.yaml' 2026-05-11T15:53:34,255 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_disputes.yaml' 2026-05-11T15:53:34,256 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_moral_scenarios.yaml' 2026-05-11T15:53:34,258 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_nutrition.yaml' 2026-05-11T15:53:34,259 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_philosophy.yaml' 2026-05-11T15:53:34,260 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_prehistory.yaml' 2026-05-11T15:53:34,261 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_accounting.yaml' 2026-05-11T15:53:34,262 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_law.yaml' 2026-05-11T15:53:34,263 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_medicine.yaml' 2026-05-11T15:53:34,265 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_professional_psychology.yaml' 2026-05-11T15:53:34,266 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_public_relations.yaml' 2026-05-11T15:53:34,267 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_security_studies.yaml' 2026-05-11T15:53:34,268 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_sociology.yaml' 2026-05-11T15:53:34,269 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_us_foreign_policy.yaml' 2026-05-11T15:53:34,270 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_virology.yaml' 2026-05-11T15:53:34,271 adding 'lm_eval/tasks/global_mmlu/full/ny/global_mmlu_full_ny_world_religions.yaml' 2026-05-11T15:53:34,272 adding 'lm_eval/tasks/global_mmlu/full/ny/utils.py' 2026-05-11T15:53:34,275 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl.yaml' 2026-05-11T15:53:34,276 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_humanities.yaml' 2026-05-11T15:53:34,277 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_other.yaml' 2026-05-11T15:53:34,279 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_social_sciences.yaml' 2026-05-11T15:53:34,280 adding 'lm_eval/tasks/global_mmlu/full/pl/_global_mmlu_full_pl_stem.yaml' 2026-05-11T15:53:34,281 adding 'lm_eval/tasks/global_mmlu/full/pl/_pl_template_yaml' 2026-05-11T15:53:34,282 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_abstract_algebra.yaml' 2026-05-11T15:53:34,283 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_anatomy.yaml' 2026-05-11T15:53:34,284 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_astronomy.yaml' 2026-05-11T15:53:34,286 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_business_ethics.yaml' 2026-05-11T15:53:34,287 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_clinical_knowledge.yaml' 2026-05-11T15:53:34,288 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_biology.yaml' 2026-05-11T15:53:34,289 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_chemistry.yaml' 2026-05-11T15:53:34,290 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_computer_science.yaml' 2026-05-11T15:53:34,292 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_mathematics.yaml' 2026-05-11T15:53:34,293 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_medicine.yaml' 2026-05-11T15:53:34,294 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_college_physics.yaml' 2026-05-11T15:53:34,295 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_computer_security.yaml' 2026-05-11T15:53:34,296 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_conceptual_physics.yaml' 2026-05-11T15:53:34,298 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_econometrics.yaml' 2026-05-11T15:53:34,299 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_electrical_engineering.yaml' 2026-05-11T15:53:34,300 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_elementary_mathematics.yaml' 2026-05-11T15:53:34,301 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_formal_logic.yaml' 2026-05-11T15:53:34,302 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_global_facts.yaml' 2026-05-11T15:53:34,304 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_biology.yaml' 2026-05-11T15:53:34,305 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_chemistry.yaml' 2026-05-11T15:53:34,306 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_computer_science.yaml' 2026-05-11T15:53:34,307 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_european_history.yaml' 2026-05-11T15:53:34,308 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_geography.yaml' 2026-05-11T15:53:34,309 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,311 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,312 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_mathematics.yaml' 2026-05-11T15:53:34,313 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_microeconomics.yaml' 2026-05-11T15:53:34,314 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_physics.yaml' 2026-05-11T15:53:34,315 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_psychology.yaml' 2026-05-11T15:53:34,316 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_statistics.yaml' 2026-05-11T15:53:34,317 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_us_history.yaml' 2026-05-11T15:53:34,318 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_high_school_world_history.yaml' 2026-05-11T15:53:34,320 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_aging.yaml' 2026-05-11T15:53:34,321 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_human_sexuality.yaml' 2026-05-11T15:53:34,322 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_international_law.yaml' 2026-05-11T15:53:34,323 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_jurisprudence.yaml' 2026-05-11T15:53:34,324 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_logical_fallacies.yaml' 2026-05-11T15:53:34,325 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_machine_learning.yaml' 2026-05-11T15:53:34,327 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_management.yaml' 2026-05-11T15:53:34,328 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_marketing.yaml' 2026-05-11T15:53:34,329 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_medical_genetics.yaml' 2026-05-11T15:53:34,330 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_miscellaneous.yaml' 2026-05-11T15:53:34,331 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_disputes.yaml' 2026-05-11T15:53:34,332 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_moral_scenarios.yaml' 2026-05-11T15:53:34,333 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_nutrition.yaml' 2026-05-11T15:53:34,335 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_philosophy.yaml' 2026-05-11T15:53:34,336 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_prehistory.yaml' 2026-05-11T15:53:34,337 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_accounting.yaml' 2026-05-11T15:53:34,338 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_law.yaml' 2026-05-11T15:53:34,339 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_medicine.yaml' 2026-05-11T15:53:34,341 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_professional_psychology.yaml' 2026-05-11T15:53:34,342 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_public_relations.yaml' 2026-05-11T15:53:34,343 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_security_studies.yaml' 2026-05-11T15:53:34,344 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_sociology.yaml' 2026-05-11T15:53:34,345 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_us_foreign_policy.yaml' 2026-05-11T15:53:34,346 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_virology.yaml' 2026-05-11T15:53:34,347 adding 'lm_eval/tasks/global_mmlu/full/pl/global_mmlu_full_pl_world_religions.yaml' 2026-05-11T15:53:34,349 adding 'lm_eval/tasks/global_mmlu/full/pl/utils.py' 2026-05-11T15:53:34,352 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt.yaml' 2026-05-11T15:53:34,353 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_humanities.yaml' 2026-05-11T15:53:34,354 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_other.yaml' 2026-05-11T15:53:34,355 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_social_sciences.yaml' 2026-05-11T15:53:34,356 adding 'lm_eval/tasks/global_mmlu/full/pt/_global_mmlu_full_pt_stem.yaml' 2026-05-11T15:53:34,357 adding 'lm_eval/tasks/global_mmlu/full/pt/_pt_template_yaml' 2026-05-11T15:53:34,358 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_abstract_algebra.yaml' 2026-05-11T15:53:34,359 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_anatomy.yaml' 2026-05-11T15:53:34,361 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_astronomy.yaml' 2026-05-11T15:53:34,362 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_business_ethics.yaml' 2026-05-11T15:53:34,363 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_clinical_knowledge.yaml' 2026-05-11T15:53:34,364 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_biology.yaml' 2026-05-11T15:53:34,365 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_chemistry.yaml' 2026-05-11T15:53:34,366 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_computer_science.yaml' 2026-05-11T15:53:34,368 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_mathematics.yaml' 2026-05-11T15:53:34,369 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_medicine.yaml' 2026-05-11T15:53:34,370 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_college_physics.yaml' 2026-05-11T15:53:34,371 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_computer_security.yaml' 2026-05-11T15:53:34,372 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_conceptual_physics.yaml' 2026-05-11T15:53:34,373 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_econometrics.yaml' 2026-05-11T15:53:34,374 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_electrical_engineering.yaml' 2026-05-11T15:53:34,376 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_elementary_mathematics.yaml' 2026-05-11T15:53:34,377 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_formal_logic.yaml' 2026-05-11T15:53:34,378 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_global_facts.yaml' 2026-05-11T15:53:34,379 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_biology.yaml' 2026-05-11T15:53:34,380 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_chemistry.yaml' 2026-05-11T15:53:34,382 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_computer_science.yaml' 2026-05-11T15:53:34,383 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_european_history.yaml' 2026-05-11T15:53:34,384 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_geography.yaml' 2026-05-11T15:53:34,385 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,386 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,387 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_mathematics.yaml' 2026-05-11T15:53:34,389 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_microeconomics.yaml' 2026-05-11T15:53:34,390 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_physics.yaml' 2026-05-11T15:53:34,391 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_psychology.yaml' 2026-05-11T15:53:34,393 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_statistics.yaml' 2026-05-11T15:53:34,394 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_us_history.yaml' 2026-05-11T15:53:34,395 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_high_school_world_history.yaml' 2026-05-11T15:53:34,396 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_aging.yaml' 2026-05-11T15:53:34,397 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_human_sexuality.yaml' 2026-05-11T15:53:34,398 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_international_law.yaml' 2026-05-11T15:53:34,399 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_jurisprudence.yaml' 2026-05-11T15:53:34,400 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_logical_fallacies.yaml' 2026-05-11T15:53:34,401 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_machine_learning.yaml' 2026-05-11T15:53:34,402 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_management.yaml' 2026-05-11T15:53:34,404 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_marketing.yaml' 2026-05-11T15:53:34,405 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_medical_genetics.yaml' 2026-05-11T15:53:34,406 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_miscellaneous.yaml' 2026-05-11T15:53:34,407 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_disputes.yaml' 2026-05-11T15:53:34,408 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_moral_scenarios.yaml' 2026-05-11T15:53:34,410 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_nutrition.yaml' 2026-05-11T15:53:34,411 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_philosophy.yaml' 2026-05-11T15:53:34,412 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_prehistory.yaml' 2026-05-11T15:53:34,413 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_accounting.yaml' 2026-05-11T15:53:34,414 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_law.yaml' 2026-05-11T15:53:34,415 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_medicine.yaml' 2026-05-11T15:53:34,416 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_professional_psychology.yaml' 2026-05-11T15:53:34,417 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_public_relations.yaml' 2026-05-11T15:53:34,419 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_security_studies.yaml' 2026-05-11T15:53:34,420 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_sociology.yaml' 2026-05-11T15:53:34,421 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_us_foreign_policy.yaml' 2026-05-11T15:53:34,422 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_virology.yaml' 2026-05-11T15:53:34,423 adding 'lm_eval/tasks/global_mmlu/full/pt/global_mmlu_full_pt_world_religions.yaml' 2026-05-11T15:53:34,424 adding 'lm_eval/tasks/global_mmlu/full/pt/utils.py' 2026-05-11T15:53:34,428 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro.yaml' 2026-05-11T15:53:34,429 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_humanities.yaml' 2026-05-11T15:53:34,430 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_other.yaml' 2026-05-11T15:53:34,431 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_social_sciences.yaml' 2026-05-11T15:53:34,432 adding 'lm_eval/tasks/global_mmlu/full/ro/_global_mmlu_full_ro_stem.yaml' 2026-05-11T15:53:34,433 adding 'lm_eval/tasks/global_mmlu/full/ro/_ro_template_yaml' 2026-05-11T15:53:34,435 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_abstract_algebra.yaml' 2026-05-11T15:53:34,436 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_anatomy.yaml' 2026-05-11T15:53:34,437 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_astronomy.yaml' 2026-05-11T15:53:34,438 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_business_ethics.yaml' 2026-05-11T15:53:34,439 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_clinical_knowledge.yaml' 2026-05-11T15:53:34,440 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_biology.yaml' 2026-05-11T15:53:34,442 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_chemistry.yaml' 2026-05-11T15:53:34,443 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_computer_science.yaml' 2026-05-11T15:53:34,444 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_mathematics.yaml' 2026-05-11T15:53:34,445 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_medicine.yaml' 2026-05-11T15:53:34,446 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_college_physics.yaml' 2026-05-11T15:53:34,447 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_computer_security.yaml' 2026-05-11T15:53:34,448 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_conceptual_physics.yaml' 2026-05-11T15:53:34,449 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_econometrics.yaml' 2026-05-11T15:53:34,451 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_electrical_engineering.yaml' 2026-05-11T15:53:34,452 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_elementary_mathematics.yaml' 2026-05-11T15:53:34,453 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_formal_logic.yaml' 2026-05-11T15:53:34,454 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_global_facts.yaml' 2026-05-11T15:53:34,455 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_biology.yaml' 2026-05-11T15:53:34,456 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_chemistry.yaml' 2026-05-11T15:53:34,458 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_computer_science.yaml' 2026-05-11T15:53:34,459 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_european_history.yaml' 2026-05-11T15:53:34,460 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_geography.yaml' 2026-05-11T15:53:34,461 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,462 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,463 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_mathematics.yaml' 2026-05-11T15:53:34,465 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_microeconomics.yaml' 2026-05-11T15:53:34,466 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_physics.yaml' 2026-05-11T15:53:34,467 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_psychology.yaml' 2026-05-11T15:53:34,468 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_statistics.yaml' 2026-05-11T15:53:34,469 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_us_history.yaml' 2026-05-11T15:53:34,471 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_high_school_world_history.yaml' 2026-05-11T15:53:34,472 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_aging.yaml' 2026-05-11T15:53:34,473 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_human_sexuality.yaml' 2026-05-11T15:53:34,474 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_international_law.yaml' 2026-05-11T15:53:34,475 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_jurisprudence.yaml' 2026-05-11T15:53:34,476 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_logical_fallacies.yaml' 2026-05-11T15:53:34,478 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_machine_learning.yaml' 2026-05-11T15:53:34,479 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_management.yaml' 2026-05-11T15:53:34,480 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_marketing.yaml' 2026-05-11T15:53:34,481 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_medical_genetics.yaml' 2026-05-11T15:53:34,482 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_miscellaneous.yaml' 2026-05-11T15:53:34,483 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_disputes.yaml' 2026-05-11T15:53:34,485 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_moral_scenarios.yaml' 2026-05-11T15:53:34,486 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_nutrition.yaml' 2026-05-11T15:53:34,487 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_philosophy.yaml' 2026-05-11T15:53:34,488 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_prehistory.yaml' 2026-05-11T15:53:34,489 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_accounting.yaml' 2026-05-11T15:53:34,490 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_law.yaml' 2026-05-11T15:53:34,491 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_medicine.yaml' 2026-05-11T15:53:34,492 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_professional_psychology.yaml' 2026-05-11T15:53:34,493 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_public_relations.yaml' 2026-05-11T15:53:34,495 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_security_studies.yaml' 2026-05-11T15:53:34,496 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_sociology.yaml' 2026-05-11T15:53:34,497 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_us_foreign_policy.yaml' 2026-05-11T15:53:34,498 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_virology.yaml' 2026-05-11T15:53:34,499 adding 'lm_eval/tasks/global_mmlu/full/ro/global_mmlu_full_ro_world_religions.yaml' 2026-05-11T15:53:34,500 adding 'lm_eval/tasks/global_mmlu/full/ro/utils.py' 2026-05-11T15:53:34,503 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru.yaml' 2026-05-11T15:53:34,504 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_humanities.yaml' 2026-05-11T15:53:34,506 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_other.yaml' 2026-05-11T15:53:34,507 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_social_sciences.yaml' 2026-05-11T15:53:34,508 adding 'lm_eval/tasks/global_mmlu/full/ru/_global_mmlu_full_ru_stem.yaml' 2026-05-11T15:53:34,509 adding 'lm_eval/tasks/global_mmlu/full/ru/_ru_template_yaml' 2026-05-11T15:53:34,510 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_abstract_algebra.yaml' 2026-05-11T15:53:34,511 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_anatomy.yaml' 2026-05-11T15:53:34,513 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_astronomy.yaml' 2026-05-11T15:53:34,514 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_business_ethics.yaml' 2026-05-11T15:53:34,515 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_clinical_knowledge.yaml' 2026-05-11T15:53:34,516 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_biology.yaml' 2026-05-11T15:53:34,517 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_chemistry.yaml' 2026-05-11T15:53:34,518 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_computer_science.yaml' 2026-05-11T15:53:34,520 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_mathematics.yaml' 2026-05-11T15:53:34,521 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_medicine.yaml' 2026-05-11T15:53:34,522 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_college_physics.yaml' 2026-05-11T15:53:34,523 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_computer_security.yaml' 2026-05-11T15:53:34,524 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_conceptual_physics.yaml' 2026-05-11T15:53:34,525 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_econometrics.yaml' 2026-05-11T15:53:34,527 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_electrical_engineering.yaml' 2026-05-11T15:53:34,528 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_elementary_mathematics.yaml' 2026-05-11T15:53:34,529 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_formal_logic.yaml' 2026-05-11T15:53:34,530 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_global_facts.yaml' 2026-05-11T15:53:34,531 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_biology.yaml' 2026-05-11T15:53:34,532 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_chemistry.yaml' 2026-05-11T15:53:34,533 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_computer_science.yaml' 2026-05-11T15:53:34,534 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_european_history.yaml' 2026-05-11T15:53:34,535 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_geography.yaml' 2026-05-11T15:53:34,536 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,538 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,539 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_mathematics.yaml' 2026-05-11T15:53:34,540 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_microeconomics.yaml' 2026-05-11T15:53:34,541 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_physics.yaml' 2026-05-11T15:53:34,542 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_psychology.yaml' 2026-05-11T15:53:34,543 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_statistics.yaml' 2026-05-11T15:53:34,544 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_us_history.yaml' 2026-05-11T15:53:34,546 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_high_school_world_history.yaml' 2026-05-11T15:53:34,547 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_aging.yaml' 2026-05-11T15:53:34,548 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_human_sexuality.yaml' 2026-05-11T15:53:34,549 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_international_law.yaml' 2026-05-11T15:53:34,550 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_jurisprudence.yaml' 2026-05-11T15:53:34,551 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_logical_fallacies.yaml' 2026-05-11T15:53:34,553 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_machine_learning.yaml' 2026-05-11T15:53:34,554 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_management.yaml' 2026-05-11T15:53:34,555 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_marketing.yaml' 2026-05-11T15:53:34,556 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_medical_genetics.yaml' 2026-05-11T15:53:34,557 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_miscellaneous.yaml' 2026-05-11T15:53:34,558 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_disputes.yaml' 2026-05-11T15:53:34,560 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_moral_scenarios.yaml' 2026-05-11T15:53:34,561 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_nutrition.yaml' 2026-05-11T15:53:34,562 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_philosophy.yaml' 2026-05-11T15:53:34,563 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_prehistory.yaml' 2026-05-11T15:53:34,564 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_accounting.yaml' 2026-05-11T15:53:34,565 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_law.yaml' 2026-05-11T15:53:34,567 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_medicine.yaml' 2026-05-11T15:53:34,568 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_professional_psychology.yaml' 2026-05-11T15:53:34,569 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_public_relations.yaml' 2026-05-11T15:53:34,570 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_security_studies.yaml' 2026-05-11T15:53:34,571 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_sociology.yaml' 2026-05-11T15:53:34,572 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_us_foreign_policy.yaml' 2026-05-11T15:53:34,573 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_virology.yaml' 2026-05-11T15:53:34,574 adding 'lm_eval/tasks/global_mmlu/full/ru/global_mmlu_full_ru_world_religions.yaml' 2026-05-11T15:53:34,576 adding 'lm_eval/tasks/global_mmlu/full/ru/utils.py' 2026-05-11T15:53:34,578 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si.yaml' 2026-05-11T15:53:34,580 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_humanities.yaml' 2026-05-11T15:53:34,581 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_other.yaml' 2026-05-11T15:53:34,582 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_social_sciences.yaml' 2026-05-11T15:53:34,583 adding 'lm_eval/tasks/global_mmlu/full/si/_global_mmlu_full_si_stem.yaml' 2026-05-11T15:53:34,584 adding 'lm_eval/tasks/global_mmlu/full/si/_si_template_yaml' 2026-05-11T15:53:34,585 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_abstract_algebra.yaml' 2026-05-11T15:53:34,586 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_anatomy.yaml' 2026-05-11T15:53:34,588 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_astronomy.yaml' 2026-05-11T15:53:34,589 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_business_ethics.yaml' 2026-05-11T15:53:34,590 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_clinical_knowledge.yaml' 2026-05-11T15:53:34,591 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_biology.yaml' 2026-05-11T15:53:34,592 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_chemistry.yaml' 2026-05-11T15:53:34,594 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_computer_science.yaml' 2026-05-11T15:53:34,595 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_mathematics.yaml' 2026-05-11T15:53:34,596 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_medicine.yaml' 2026-05-11T15:53:34,597 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_college_physics.yaml' 2026-05-11T15:53:34,599 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_computer_security.yaml' 2026-05-11T15:53:34,600 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_conceptual_physics.yaml' 2026-05-11T15:53:34,601 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_econometrics.yaml' 2026-05-11T15:53:34,602 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_electrical_engineering.yaml' 2026-05-11T15:53:34,603 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_elementary_mathematics.yaml' 2026-05-11T15:53:34,604 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_formal_logic.yaml' 2026-05-11T15:53:34,606 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_global_facts.yaml' 2026-05-11T15:53:34,607 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_biology.yaml' 2026-05-11T15:53:34,608 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_chemistry.yaml' 2026-05-11T15:53:34,609 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_computer_science.yaml' 2026-05-11T15:53:34,610 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_european_history.yaml' 2026-05-11T15:53:34,611 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_geography.yaml' 2026-05-11T15:53:34,613 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,614 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,615 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_mathematics.yaml' 2026-05-11T15:53:34,616 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_microeconomics.yaml' 2026-05-11T15:53:34,617 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_physics.yaml' 2026-05-11T15:53:34,618 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_psychology.yaml' 2026-05-11T15:53:34,619 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_statistics.yaml' 2026-05-11T15:53:34,620 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_us_history.yaml' 2026-05-11T15:53:34,621 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_high_school_world_history.yaml' 2026-05-11T15:53:34,623 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_aging.yaml' 2026-05-11T15:53:34,624 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_human_sexuality.yaml' 2026-05-11T15:53:34,625 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_international_law.yaml' 2026-05-11T15:53:34,626 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_jurisprudence.yaml' 2026-05-11T15:53:34,627 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_logical_fallacies.yaml' 2026-05-11T15:53:34,628 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_machine_learning.yaml' 2026-05-11T15:53:34,630 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_management.yaml' 2026-05-11T15:53:34,631 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_marketing.yaml' 2026-05-11T15:53:34,632 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_medical_genetics.yaml' 2026-05-11T15:53:34,633 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_miscellaneous.yaml' 2026-05-11T15:53:34,634 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_disputes.yaml' 2026-05-11T15:53:34,635 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_moral_scenarios.yaml' 2026-05-11T15:53:34,636 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_nutrition.yaml' 2026-05-11T15:53:34,638 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_philosophy.yaml' 2026-05-11T15:53:34,639 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_prehistory.yaml' 2026-05-11T15:53:34,640 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_accounting.yaml' 2026-05-11T15:53:34,641 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_law.yaml' 2026-05-11T15:53:34,642 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_medicine.yaml' 2026-05-11T15:53:34,644 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_professional_psychology.yaml' 2026-05-11T15:53:34,645 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_public_relations.yaml' 2026-05-11T15:53:34,646 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_security_studies.yaml' 2026-05-11T15:53:34,647 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_sociology.yaml' 2026-05-11T15:53:34,648 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_us_foreign_policy.yaml' 2026-05-11T15:53:34,649 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_virology.yaml' 2026-05-11T15:53:34,651 adding 'lm_eval/tasks/global_mmlu/full/si/global_mmlu_full_si_world_religions.yaml' 2026-05-11T15:53:34,652 adding 'lm_eval/tasks/global_mmlu/full/si/utils.py' 2026-05-11T15:53:34,655 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn.yaml' 2026-05-11T15:53:34,656 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_humanities.yaml' 2026-05-11T15:53:34,658 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_other.yaml' 2026-05-11T15:53:34,659 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_social_sciences.yaml' 2026-05-11T15:53:34,660 adding 'lm_eval/tasks/global_mmlu/full/sn/_global_mmlu_full_sn_stem.yaml' 2026-05-11T15:53:34,661 adding 'lm_eval/tasks/global_mmlu/full/sn/_sn_template_yaml' 2026-05-11T15:53:34,662 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_abstract_algebra.yaml' 2026-05-11T15:53:34,663 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_anatomy.yaml' 2026-05-11T15:53:34,664 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_astronomy.yaml' 2026-05-11T15:53:34,665 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_business_ethics.yaml' 2026-05-11T15:53:34,666 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_clinical_knowledge.yaml' 2026-05-11T15:53:34,668 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_biology.yaml' 2026-05-11T15:53:34,669 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_chemistry.yaml' 2026-05-11T15:53:34,670 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_computer_science.yaml' 2026-05-11T15:53:34,671 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_mathematics.yaml' 2026-05-11T15:53:34,672 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_medicine.yaml' 2026-05-11T15:53:34,673 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_college_physics.yaml' 2026-05-11T15:53:34,674 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_computer_security.yaml' 2026-05-11T15:53:34,676 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_conceptual_physics.yaml' 2026-05-11T15:53:34,677 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_econometrics.yaml' 2026-05-11T15:53:34,678 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_electrical_engineering.yaml' 2026-05-11T15:53:34,679 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_elementary_mathematics.yaml' 2026-05-11T15:53:34,680 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_formal_logic.yaml' 2026-05-11T15:53:34,681 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_global_facts.yaml' 2026-05-11T15:53:34,682 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_biology.yaml' 2026-05-11T15:53:34,684 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_chemistry.yaml' 2026-05-11T15:53:34,685 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_computer_science.yaml' 2026-05-11T15:53:34,686 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_european_history.yaml' 2026-05-11T15:53:34,687 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_geography.yaml' 2026-05-11T15:53:34,688 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,689 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,691 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_mathematics.yaml' 2026-05-11T15:53:34,692 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_microeconomics.yaml' 2026-05-11T15:53:34,693 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_physics.yaml' 2026-05-11T15:53:34,694 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_psychology.yaml' 2026-05-11T15:53:34,695 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_statistics.yaml' 2026-05-11T15:53:34,696 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_us_history.yaml' 2026-05-11T15:53:34,698 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_high_school_world_history.yaml' 2026-05-11T15:53:34,699 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_aging.yaml' 2026-05-11T15:53:34,700 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_human_sexuality.yaml' 2026-05-11T15:53:34,701 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_international_law.yaml' 2026-05-11T15:53:34,702 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_jurisprudence.yaml' 2026-05-11T15:53:34,703 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_logical_fallacies.yaml' 2026-05-11T15:53:34,704 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_machine_learning.yaml' 2026-05-11T15:53:34,705 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_management.yaml' 2026-05-11T15:53:34,707 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_marketing.yaml' 2026-05-11T15:53:34,708 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_medical_genetics.yaml' 2026-05-11T15:53:34,709 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_miscellaneous.yaml' 2026-05-11T15:53:34,710 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_disputes.yaml' 2026-05-11T15:53:34,711 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_moral_scenarios.yaml' 2026-05-11T15:53:34,713 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_nutrition.yaml' 2026-05-11T15:53:34,714 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_philosophy.yaml' 2026-05-11T15:53:34,715 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_prehistory.yaml' 2026-05-11T15:53:34,716 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_accounting.yaml' 2026-05-11T15:53:34,717 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_law.yaml' 2026-05-11T15:53:34,718 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_medicine.yaml' 2026-05-11T15:53:34,719 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_professional_psychology.yaml' 2026-05-11T15:53:34,720 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_public_relations.yaml' 2026-05-11T15:53:34,722 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_security_studies.yaml' 2026-05-11T15:53:34,723 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_sociology.yaml' 2026-05-11T15:53:34,724 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_us_foreign_policy.yaml' 2026-05-11T15:53:34,725 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_virology.yaml' 2026-05-11T15:53:34,726 adding 'lm_eval/tasks/global_mmlu/full/sn/global_mmlu_full_sn_world_religions.yaml' 2026-05-11T15:53:34,728 adding 'lm_eval/tasks/global_mmlu/full/sn/utils.py' 2026-05-11T15:53:34,731 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so.yaml' 2026-05-11T15:53:34,732 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_humanities.yaml' 2026-05-11T15:53:34,733 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_other.yaml' 2026-05-11T15:53:34,734 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_social_sciences.yaml' 2026-05-11T15:53:34,735 adding 'lm_eval/tasks/global_mmlu/full/so/_global_mmlu_full_so_stem.yaml' 2026-05-11T15:53:34,737 adding 'lm_eval/tasks/global_mmlu/full/so/_so_template_yaml' 2026-05-11T15:53:34,738 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_abstract_algebra.yaml' 2026-05-11T15:53:34,739 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_anatomy.yaml' 2026-05-11T15:53:34,740 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_astronomy.yaml' 2026-05-11T15:53:34,741 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_business_ethics.yaml' 2026-05-11T15:53:34,743 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_clinical_knowledge.yaml' 2026-05-11T15:53:34,744 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_biology.yaml' 2026-05-11T15:53:34,745 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_chemistry.yaml' 2026-05-11T15:53:34,746 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_computer_science.yaml' 2026-05-11T15:53:34,747 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_mathematics.yaml' 2026-05-11T15:53:34,748 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_medicine.yaml' 2026-05-11T15:53:34,750 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_college_physics.yaml' 2026-05-11T15:53:34,751 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_computer_security.yaml' 2026-05-11T15:53:34,752 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_conceptual_physics.yaml' 2026-05-11T15:53:34,753 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_econometrics.yaml' 2026-05-11T15:53:34,754 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_electrical_engineering.yaml' 2026-05-11T15:53:34,755 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_elementary_mathematics.yaml' 2026-05-11T15:53:34,756 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_formal_logic.yaml' 2026-05-11T15:53:34,757 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_global_facts.yaml' 2026-05-11T15:53:34,758 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_biology.yaml' 2026-05-11T15:53:34,760 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_chemistry.yaml' 2026-05-11T15:53:34,761 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_computer_science.yaml' 2026-05-11T15:53:34,762 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_european_history.yaml' 2026-05-11T15:53:34,763 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_geography.yaml' 2026-05-11T15:53:34,764 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,765 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,767 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_mathematics.yaml' 2026-05-11T15:53:34,768 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_microeconomics.yaml' 2026-05-11T15:53:34,769 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_physics.yaml' 2026-05-11T15:53:34,770 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_psychology.yaml' 2026-05-11T15:53:34,771 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_statistics.yaml' 2026-05-11T15:53:34,772 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_us_history.yaml' 2026-05-11T15:53:34,773 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_high_school_world_history.yaml' 2026-05-11T15:53:34,775 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_aging.yaml' 2026-05-11T15:53:34,776 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_human_sexuality.yaml' 2026-05-11T15:53:34,777 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_international_law.yaml' 2026-05-11T15:53:34,778 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_jurisprudence.yaml' 2026-05-11T15:53:34,779 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_logical_fallacies.yaml' 2026-05-11T15:53:34,780 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_machine_learning.yaml' 2026-05-11T15:53:34,782 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_management.yaml' 2026-05-11T15:53:34,783 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_marketing.yaml' 2026-05-11T15:53:34,784 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_medical_genetics.yaml' 2026-05-11T15:53:34,785 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_miscellaneous.yaml' 2026-05-11T15:53:34,786 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_disputes.yaml' 2026-05-11T15:53:34,787 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_moral_scenarios.yaml' 2026-05-11T15:53:34,789 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_nutrition.yaml' 2026-05-11T15:53:34,790 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_philosophy.yaml' 2026-05-11T15:53:34,791 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_prehistory.yaml' 2026-05-11T15:53:34,792 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_accounting.yaml' 2026-05-11T15:53:34,793 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_law.yaml' 2026-05-11T15:53:34,794 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_medicine.yaml' 2026-05-11T15:53:34,795 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_professional_psychology.yaml' 2026-05-11T15:53:34,796 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_public_relations.yaml' 2026-05-11T15:53:34,797 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_security_studies.yaml' 2026-05-11T15:53:34,799 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_sociology.yaml' 2026-05-11T15:53:34,800 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_us_foreign_policy.yaml' 2026-05-11T15:53:34,801 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_virology.yaml' 2026-05-11T15:53:34,802 adding 'lm_eval/tasks/global_mmlu/full/so/global_mmlu_full_so_world_religions.yaml' 2026-05-11T15:53:34,803 adding 'lm_eval/tasks/global_mmlu/full/so/utils.py' 2026-05-11T15:53:34,806 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr.yaml' 2026-05-11T15:53:34,807 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_humanities.yaml' 2026-05-11T15:53:34,809 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_other.yaml' 2026-05-11T15:53:34,810 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_social_sciences.yaml' 2026-05-11T15:53:34,811 adding 'lm_eval/tasks/global_mmlu/full/sr/_global_mmlu_full_sr_stem.yaml' 2026-05-11T15:53:34,812 adding 'lm_eval/tasks/global_mmlu/full/sr/_sr_template_yaml' 2026-05-11T15:53:34,813 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_abstract_algebra.yaml' 2026-05-11T15:53:34,815 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_anatomy.yaml' 2026-05-11T15:53:34,816 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_astronomy.yaml' 2026-05-11T15:53:34,817 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_business_ethics.yaml' 2026-05-11T15:53:34,818 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_clinical_knowledge.yaml' 2026-05-11T15:53:34,819 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_biology.yaml' 2026-05-11T15:53:34,820 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_chemistry.yaml' 2026-05-11T15:53:34,822 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_computer_science.yaml' 2026-05-11T15:53:34,823 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_mathematics.yaml' 2026-05-11T15:53:34,824 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_medicine.yaml' 2026-05-11T15:53:34,825 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_college_physics.yaml' 2026-05-11T15:53:34,826 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_computer_security.yaml' 2026-05-11T15:53:34,828 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_conceptual_physics.yaml' 2026-05-11T15:53:34,829 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_econometrics.yaml' 2026-05-11T15:53:34,830 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_electrical_engineering.yaml' 2026-05-11T15:53:34,831 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_elementary_mathematics.yaml' 2026-05-11T15:53:34,833 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_formal_logic.yaml' 2026-05-11T15:53:34,834 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_global_facts.yaml' 2026-05-11T15:53:34,835 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_biology.yaml' 2026-05-11T15:53:34,836 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_chemistry.yaml' 2026-05-11T15:53:34,837 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_computer_science.yaml' 2026-05-11T15:53:34,839 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_european_history.yaml' 2026-05-11T15:53:34,840 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_geography.yaml' 2026-05-11T15:53:34,841 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,842 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,843 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_mathematics.yaml' 2026-05-11T15:53:34,844 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_microeconomics.yaml' 2026-05-11T15:53:34,845 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_physics.yaml' 2026-05-11T15:53:34,847 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_psychology.yaml' 2026-05-11T15:53:34,848 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_statistics.yaml' 2026-05-11T15:53:34,849 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_us_history.yaml' 2026-05-11T15:53:34,850 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_high_school_world_history.yaml' 2026-05-11T15:53:34,851 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_aging.yaml' 2026-05-11T15:53:34,852 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_human_sexuality.yaml' 2026-05-11T15:53:34,853 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_international_law.yaml' 2026-05-11T15:53:34,854 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_jurisprudence.yaml' 2026-05-11T15:53:34,856 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_logical_fallacies.yaml' 2026-05-11T15:53:34,857 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_machine_learning.yaml' 2026-05-11T15:53:34,858 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_management.yaml' 2026-05-11T15:53:34,859 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_marketing.yaml' 2026-05-11T15:53:34,860 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_medical_genetics.yaml' 2026-05-11T15:53:34,861 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_miscellaneous.yaml' 2026-05-11T15:53:34,863 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_disputes.yaml' 2026-05-11T15:53:34,864 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_moral_scenarios.yaml' 2026-05-11T15:53:34,865 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_nutrition.yaml' 2026-05-11T15:53:34,866 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_philosophy.yaml' 2026-05-11T15:53:34,868 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_prehistory.yaml' 2026-05-11T15:53:34,869 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_accounting.yaml' 2026-05-11T15:53:34,870 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_law.yaml' 2026-05-11T15:53:34,871 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_medicine.yaml' 2026-05-11T15:53:34,872 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_professional_psychology.yaml' 2026-05-11T15:53:34,874 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_public_relations.yaml' 2026-05-11T15:53:34,875 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_security_studies.yaml' 2026-05-11T15:53:34,876 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_sociology.yaml' 2026-05-11T15:53:34,877 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_us_foreign_policy.yaml' 2026-05-11T15:53:34,878 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_virology.yaml' 2026-05-11T15:53:34,879 adding 'lm_eval/tasks/global_mmlu/full/sr/global_mmlu_full_sr_world_religions.yaml' 2026-05-11T15:53:34,881 adding 'lm_eval/tasks/global_mmlu/full/sr/utils.py' 2026-05-11T15:53:34,884 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv.yaml' 2026-05-11T15:53:34,885 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_humanities.yaml' 2026-05-11T15:53:34,886 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_other.yaml' 2026-05-11T15:53:34,887 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_social_sciences.yaml' 2026-05-11T15:53:34,888 adding 'lm_eval/tasks/global_mmlu/full/sv/_global_mmlu_full_sv_stem.yaml' 2026-05-11T15:53:34,889 adding 'lm_eval/tasks/global_mmlu/full/sv/_sv_template_yaml' 2026-05-11T15:53:34,890 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_abstract_algebra.yaml' 2026-05-11T15:53:34,891 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_anatomy.yaml' 2026-05-11T15:53:34,892 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_astronomy.yaml' 2026-05-11T15:53:34,894 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_business_ethics.yaml' 2026-05-11T15:53:34,895 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_clinical_knowledge.yaml' 2026-05-11T15:53:34,896 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_biology.yaml' 2026-05-11T15:53:34,897 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_chemistry.yaml' 2026-05-11T15:53:34,898 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_computer_science.yaml' 2026-05-11T15:53:34,899 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_mathematics.yaml' 2026-05-11T15:53:34,900 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_medicine.yaml' 2026-05-11T15:53:34,901 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_college_physics.yaml' 2026-05-11T15:53:34,903 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_computer_security.yaml' 2026-05-11T15:53:34,904 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_conceptual_physics.yaml' 2026-05-11T15:53:34,905 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_econometrics.yaml' 2026-05-11T15:53:34,906 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_electrical_engineering.yaml' 2026-05-11T15:53:34,907 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_elementary_mathematics.yaml' 2026-05-11T15:53:34,908 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_formal_logic.yaml' 2026-05-11T15:53:34,910 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_global_facts.yaml' 2026-05-11T15:53:34,911 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_biology.yaml' 2026-05-11T15:53:34,912 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_chemistry.yaml' 2026-05-11T15:53:34,913 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_computer_science.yaml' 2026-05-11T15:53:34,914 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_european_history.yaml' 2026-05-11T15:53:34,915 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_geography.yaml' 2026-05-11T15:53:34,917 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,918 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,919 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_mathematics.yaml' 2026-05-11T15:53:34,920 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_microeconomics.yaml' 2026-05-11T15:53:34,921 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_physics.yaml' 2026-05-11T15:53:34,922 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_psychology.yaml' 2026-05-11T15:53:34,924 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_statistics.yaml' 2026-05-11T15:53:34,925 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_us_history.yaml' 2026-05-11T15:53:34,926 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_high_school_world_history.yaml' 2026-05-11T15:53:34,927 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_aging.yaml' 2026-05-11T15:53:34,928 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_human_sexuality.yaml' 2026-05-11T15:53:34,929 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_international_law.yaml' 2026-05-11T15:53:34,930 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_jurisprudence.yaml' 2026-05-11T15:53:34,931 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_logical_fallacies.yaml' 2026-05-11T15:53:34,932 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_machine_learning.yaml' 2026-05-11T15:53:34,934 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_management.yaml' 2026-05-11T15:53:34,935 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_marketing.yaml' 2026-05-11T15:53:34,936 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_medical_genetics.yaml' 2026-05-11T15:53:34,937 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_miscellaneous.yaml' 2026-05-11T15:53:34,938 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_disputes.yaml' 2026-05-11T15:53:34,939 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_moral_scenarios.yaml' 2026-05-11T15:53:34,940 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_nutrition.yaml' 2026-05-11T15:53:34,941 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_philosophy.yaml' 2026-05-11T15:53:34,943 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_prehistory.yaml' 2026-05-11T15:53:34,944 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_accounting.yaml' 2026-05-11T15:53:34,945 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_law.yaml' 2026-05-11T15:53:34,946 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_medicine.yaml' 2026-05-11T15:53:34,947 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_professional_psychology.yaml' 2026-05-11T15:53:34,948 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_public_relations.yaml' 2026-05-11T15:53:34,950 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_security_studies.yaml' 2026-05-11T15:53:34,951 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_sociology.yaml' 2026-05-11T15:53:34,952 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_us_foreign_policy.yaml' 2026-05-11T15:53:34,953 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_virology.yaml' 2026-05-11T15:53:34,954 adding 'lm_eval/tasks/global_mmlu/full/sv/global_mmlu_full_sv_world_religions.yaml' 2026-05-11T15:53:34,956 adding 'lm_eval/tasks/global_mmlu/full/sv/utils.py' 2026-05-11T15:53:34,959 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw.yaml' 2026-05-11T15:53:34,960 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_humanities.yaml' 2026-05-11T15:53:34,961 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_other.yaml' 2026-05-11T15:53:34,962 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_social_sciences.yaml' 2026-05-11T15:53:34,964 adding 'lm_eval/tasks/global_mmlu/full/sw/_global_mmlu_full_sw_stem.yaml' 2026-05-11T15:53:34,965 adding 'lm_eval/tasks/global_mmlu/full/sw/_sw_template_yaml' 2026-05-11T15:53:34,966 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_abstract_algebra.yaml' 2026-05-11T15:53:34,967 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_anatomy.yaml' 2026-05-11T15:53:34,968 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_astronomy.yaml' 2026-05-11T15:53:34,969 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_business_ethics.yaml' 2026-05-11T15:53:34,971 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_clinical_knowledge.yaml' 2026-05-11T15:53:34,972 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_biology.yaml' 2026-05-11T15:53:34,973 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_chemistry.yaml' 2026-05-11T15:53:34,974 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_computer_science.yaml' 2026-05-11T15:53:34,975 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_mathematics.yaml' 2026-05-11T15:53:34,976 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_medicine.yaml' 2026-05-11T15:53:34,977 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_college_physics.yaml' 2026-05-11T15:53:34,978 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_computer_security.yaml' 2026-05-11T15:53:34,980 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_conceptual_physics.yaml' 2026-05-11T15:53:34,981 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_econometrics.yaml' 2026-05-11T15:53:34,982 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_electrical_engineering.yaml' 2026-05-11T15:53:34,983 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_elementary_mathematics.yaml' 2026-05-11T15:53:34,984 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_formal_logic.yaml' 2026-05-11T15:53:34,985 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_global_facts.yaml' 2026-05-11T15:53:34,986 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_biology.yaml' 2026-05-11T15:53:34,987 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_chemistry.yaml' 2026-05-11T15:53:34,989 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_computer_science.yaml' 2026-05-11T15:53:34,990 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_european_history.yaml' 2026-05-11T15:53:34,991 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_geography.yaml' 2026-05-11T15:53:34,992 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_government_and_politics.yaml' 2026-05-11T15:53:34,993 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_macroeconomics.yaml' 2026-05-11T15:53:34,994 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_mathematics.yaml' 2026-05-11T15:53:34,995 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_microeconomics.yaml' 2026-05-11T15:53:34,997 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_physics.yaml' 2026-05-11T15:53:34,998 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_psychology.yaml' 2026-05-11T15:53:34,999 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_statistics.yaml' 2026-05-11T15:53:35,000 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_us_history.yaml' 2026-05-11T15:53:35,001 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_high_school_world_history.yaml' 2026-05-11T15:53:35,002 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_aging.yaml' 2026-05-11T15:53:35,004 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_human_sexuality.yaml' 2026-05-11T15:53:35,005 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_international_law.yaml' 2026-05-11T15:53:35,006 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_jurisprudence.yaml' 2026-05-11T15:53:35,007 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_logical_fallacies.yaml' 2026-05-11T15:53:35,008 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_machine_learning.yaml' 2026-05-11T15:53:35,010 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_management.yaml' 2026-05-11T15:53:35,011 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_marketing.yaml' 2026-05-11T15:53:35,012 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_medical_genetics.yaml' 2026-05-11T15:53:35,013 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_miscellaneous.yaml' 2026-05-11T15:53:35,014 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_disputes.yaml' 2026-05-11T15:53:35,015 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_moral_scenarios.yaml' 2026-05-11T15:53:35,016 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_nutrition.yaml' 2026-05-11T15:53:35,017 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_philosophy.yaml' 2026-05-11T15:53:35,018 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_prehistory.yaml' 2026-05-11T15:53:35,019 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_accounting.yaml' 2026-05-11T15:53:35,021 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_law.yaml' 2026-05-11T15:53:35,022 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_medicine.yaml' 2026-05-11T15:53:35,023 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_professional_psychology.yaml' 2026-05-11T15:53:35,024 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_public_relations.yaml' 2026-05-11T15:53:35,025 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_security_studies.yaml' 2026-05-11T15:53:35,026 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_sociology.yaml' 2026-05-11T15:53:35,027 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_us_foreign_policy.yaml' 2026-05-11T15:53:35,029 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_virology.yaml' 2026-05-11T15:53:35,030 adding 'lm_eval/tasks/global_mmlu/full/sw/global_mmlu_full_sw_world_religions.yaml' 2026-05-11T15:53:35,031 adding 'lm_eval/tasks/global_mmlu/full/sw/utils.py' 2026-05-11T15:53:35,034 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te.yaml' 2026-05-11T15:53:35,035 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_humanities.yaml' 2026-05-11T15:53:35,036 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_other.yaml' 2026-05-11T15:53:35,038 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_social_sciences.yaml' 2026-05-11T15:53:35,039 adding 'lm_eval/tasks/global_mmlu/full/te/_global_mmlu_full_te_stem.yaml' 2026-05-11T15:53:35,040 adding 'lm_eval/tasks/global_mmlu/full/te/_te_template_yaml' 2026-05-11T15:53:35,041 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_abstract_algebra.yaml' 2026-05-11T15:53:35,042 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_anatomy.yaml' 2026-05-11T15:53:35,043 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_astronomy.yaml' 2026-05-11T15:53:35,045 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_business_ethics.yaml' 2026-05-11T15:53:35,046 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_clinical_knowledge.yaml' 2026-05-11T15:53:35,047 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_biology.yaml' 2026-05-11T15:53:35,048 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_chemistry.yaml' 2026-05-11T15:53:35,050 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_computer_science.yaml' 2026-05-11T15:53:35,051 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_mathematics.yaml' 2026-05-11T15:53:35,052 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_medicine.yaml' 2026-05-11T15:53:35,053 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_college_physics.yaml' 2026-05-11T15:53:35,054 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_computer_security.yaml' 2026-05-11T15:53:35,055 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_conceptual_physics.yaml' 2026-05-11T15:53:35,057 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_econometrics.yaml' 2026-05-11T15:53:35,058 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_electrical_engineering.yaml' 2026-05-11T15:53:35,059 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_elementary_mathematics.yaml' 2026-05-11T15:53:35,060 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_formal_logic.yaml' 2026-05-11T15:53:35,061 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_global_facts.yaml' 2026-05-11T15:53:35,062 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_biology.yaml' 2026-05-11T15:53:35,063 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_chemistry.yaml' 2026-05-11T15:53:35,064 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_computer_science.yaml' 2026-05-11T15:53:35,065 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_european_history.yaml' 2026-05-11T15:53:35,067 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_geography.yaml' 2026-05-11T15:53:35,068 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_government_and_politics.yaml' 2026-05-11T15:53:35,069 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_macroeconomics.yaml' 2026-05-11T15:53:35,070 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_mathematics.yaml' 2026-05-11T15:53:35,071 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_microeconomics.yaml' 2026-05-11T15:53:35,072 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_physics.yaml' 2026-05-11T15:53:35,073 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_psychology.yaml' 2026-05-11T15:53:35,075 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_statistics.yaml' 2026-05-11T15:53:35,076 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_us_history.yaml' 2026-05-11T15:53:35,077 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_high_school_world_history.yaml' 2026-05-11T15:53:35,078 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_aging.yaml' 2026-05-11T15:53:35,079 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_human_sexuality.yaml' 2026-05-11T15:53:35,080 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_international_law.yaml' 2026-05-11T15:53:35,081 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_jurisprudence.yaml' 2026-05-11T15:53:35,083 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_logical_fallacies.yaml' 2026-05-11T15:53:35,084 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_machine_learning.yaml' 2026-05-11T15:53:35,085 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_management.yaml' 2026-05-11T15:53:35,086 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_marketing.yaml' 2026-05-11T15:53:35,087 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_medical_genetics.yaml' 2026-05-11T15:53:35,088 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_miscellaneous.yaml' 2026-05-11T15:53:35,090 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_disputes.yaml' 2026-05-11T15:53:35,091 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_moral_scenarios.yaml' 2026-05-11T15:53:35,092 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_nutrition.yaml' 2026-05-11T15:53:35,093 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_philosophy.yaml' 2026-05-11T15:53:35,094 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_prehistory.yaml' 2026-05-11T15:53:35,096 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_accounting.yaml' 2026-05-11T15:53:35,097 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_law.yaml' 2026-05-11T15:53:35,098 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_medicine.yaml' 2026-05-11T15:53:35,099 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_professional_psychology.yaml' 2026-05-11T15:53:35,100 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_public_relations.yaml' 2026-05-11T15:53:35,102 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_security_studies.yaml' 2026-05-11T15:53:35,103 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_sociology.yaml' 2026-05-11T15:53:35,104 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_us_foreign_policy.yaml' 2026-05-11T15:53:35,105 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_virology.yaml' 2026-05-11T15:53:35,106 adding 'lm_eval/tasks/global_mmlu/full/te/global_mmlu_full_te_world_religions.yaml' 2026-05-11T15:53:35,107 adding 'lm_eval/tasks/global_mmlu/full/te/utils.py' 2026-05-11T15:53:35,110 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr.yaml' 2026-05-11T15:53:35,112 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_humanities.yaml' 2026-05-11T15:53:35,113 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_other.yaml' 2026-05-11T15:53:35,114 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_social_sciences.yaml' 2026-05-11T15:53:35,115 adding 'lm_eval/tasks/global_mmlu/full/tr/_global_mmlu_full_tr_stem.yaml' 2026-05-11T15:53:35,116 adding 'lm_eval/tasks/global_mmlu/full/tr/_tr_template_yaml' 2026-05-11T15:53:35,117 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_abstract_algebra.yaml' 2026-05-11T15:53:35,118 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_anatomy.yaml' 2026-05-11T15:53:35,120 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_astronomy.yaml' 2026-05-11T15:53:35,121 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_business_ethics.yaml' 2026-05-11T15:53:35,122 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_clinical_knowledge.yaml' 2026-05-11T15:53:35,123 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_biology.yaml' 2026-05-11T15:53:35,124 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_chemistry.yaml' 2026-05-11T15:53:35,126 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_computer_science.yaml' 2026-05-11T15:53:35,127 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_mathematics.yaml' 2026-05-11T15:53:35,128 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_medicine.yaml' 2026-05-11T15:53:35,129 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_college_physics.yaml' 2026-05-11T15:53:35,130 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_computer_security.yaml' 2026-05-11T15:53:35,131 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_conceptual_physics.yaml' 2026-05-11T15:53:35,132 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_econometrics.yaml' 2026-05-11T15:53:35,134 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_electrical_engineering.yaml' 2026-05-11T15:53:35,135 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_elementary_mathematics.yaml' 2026-05-11T15:53:35,136 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_formal_logic.yaml' 2026-05-11T15:53:35,137 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_global_facts.yaml' 2026-05-11T15:53:35,138 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_biology.yaml' 2026-05-11T15:53:35,140 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_chemistry.yaml' 2026-05-11T15:53:35,141 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_computer_science.yaml' 2026-05-11T15:53:35,142 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_european_history.yaml' 2026-05-11T15:53:35,143 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_geography.yaml' 2026-05-11T15:53:35,144 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_government_and_politics.yaml' 2026-05-11T15:53:35,145 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_macroeconomics.yaml' 2026-05-11T15:53:35,147 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_mathematics.yaml' 2026-05-11T15:53:35,148 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_microeconomics.yaml' 2026-05-11T15:53:35,149 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_physics.yaml' 2026-05-11T15:53:35,150 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_psychology.yaml' 2026-05-11T15:53:35,151 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_statistics.yaml' 2026-05-11T15:53:35,152 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_us_history.yaml' 2026-05-11T15:53:35,153 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_high_school_world_history.yaml' 2026-05-11T15:53:35,155 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_aging.yaml' 2026-05-11T15:53:35,156 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_human_sexuality.yaml' 2026-05-11T15:53:35,157 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_international_law.yaml' 2026-05-11T15:53:35,158 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_jurisprudence.yaml' 2026-05-11T15:53:35,159 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_logical_fallacies.yaml' 2026-05-11T15:53:35,160 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_machine_learning.yaml' 2026-05-11T15:53:35,161 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_management.yaml' 2026-05-11T15:53:35,163 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_marketing.yaml' 2026-05-11T15:53:35,164 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_medical_genetics.yaml' 2026-05-11T15:53:35,165 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_miscellaneous.yaml' 2026-05-11T15:53:35,166 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_disputes.yaml' 2026-05-11T15:53:35,167 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_moral_scenarios.yaml' 2026-05-11T15:53:35,168 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_nutrition.yaml' 2026-05-11T15:53:35,170 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_philosophy.yaml' 2026-05-11T15:53:35,171 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_prehistory.yaml' 2026-05-11T15:53:35,172 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_accounting.yaml' 2026-05-11T15:53:35,173 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_law.yaml' 2026-05-11T15:53:35,174 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_medicine.yaml' 2026-05-11T15:53:35,176 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_professional_psychology.yaml' 2026-05-11T15:53:35,177 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_public_relations.yaml' 2026-05-11T15:53:35,178 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_security_studies.yaml' 2026-05-11T15:53:35,179 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_sociology.yaml' 2026-05-11T15:53:35,180 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_us_foreign_policy.yaml' 2026-05-11T15:53:35,181 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_virology.yaml' 2026-05-11T15:53:35,183 adding 'lm_eval/tasks/global_mmlu/full/tr/global_mmlu_full_tr_world_religions.yaml' 2026-05-11T15:53:35,184 adding 'lm_eval/tasks/global_mmlu/full/tr/utils.py' 2026-05-11T15:53:35,187 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk.yaml' 2026-05-11T15:53:35,188 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_humanities.yaml' 2026-05-11T15:53:35,189 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_other.yaml' 2026-05-11T15:53:35,190 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_social_sciences.yaml' 2026-05-11T15:53:35,191 adding 'lm_eval/tasks/global_mmlu/full/uk/_global_mmlu_full_uk_stem.yaml' 2026-05-11T15:53:35,193 adding 'lm_eval/tasks/global_mmlu/full/uk/_uk_template_yaml' 2026-05-11T15:53:35,194 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_abstract_algebra.yaml' 2026-05-11T15:53:35,195 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_anatomy.yaml' 2026-05-11T15:53:35,196 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_astronomy.yaml' 2026-05-11T15:53:35,197 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_business_ethics.yaml' 2026-05-11T15:53:35,199 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_clinical_knowledge.yaml' 2026-05-11T15:53:35,200 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_biology.yaml' 2026-05-11T15:53:35,201 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_chemistry.yaml' 2026-05-11T15:53:35,202 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_computer_science.yaml' 2026-05-11T15:53:35,204 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_mathematics.yaml' 2026-05-11T15:53:35,205 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_medicine.yaml' 2026-05-11T15:53:35,206 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_college_physics.yaml' 2026-05-11T15:53:35,207 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_computer_security.yaml' 2026-05-11T15:53:35,209 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_conceptual_physics.yaml' 2026-05-11T15:53:35,210 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_econometrics.yaml' 2026-05-11T15:53:35,211 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_electrical_engineering.yaml' 2026-05-11T15:53:35,213 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_elementary_mathematics.yaml' 2026-05-11T15:53:35,214 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_formal_logic.yaml' 2026-05-11T15:53:35,215 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_global_facts.yaml' 2026-05-11T15:53:35,216 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_biology.yaml' 2026-05-11T15:53:35,217 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_chemistry.yaml' 2026-05-11T15:53:35,219 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_computer_science.yaml' 2026-05-11T15:53:35,220 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_european_history.yaml' 2026-05-11T15:53:35,221 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_geography.yaml' 2026-05-11T15:53:35,222 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_government_and_politics.yaml' 2026-05-11T15:53:35,223 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_macroeconomics.yaml' 2026-05-11T15:53:35,225 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_mathematics.yaml' 2026-05-11T15:53:35,226 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_microeconomics.yaml' 2026-05-11T15:53:35,227 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_physics.yaml' 2026-05-11T15:53:35,228 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_psychology.yaml' 2026-05-11T15:53:35,230 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_statistics.yaml' 2026-05-11T15:53:35,231 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_us_history.yaml' 2026-05-11T15:53:35,232 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_high_school_world_history.yaml' 2026-05-11T15:53:35,233 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_aging.yaml' 2026-05-11T15:53:35,234 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_human_sexuality.yaml' 2026-05-11T15:53:35,235 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_international_law.yaml' 2026-05-11T15:53:35,237 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_jurisprudence.yaml' 2026-05-11T15:53:35,238 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_logical_fallacies.yaml' 2026-05-11T15:53:35,239 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_machine_learning.yaml' 2026-05-11T15:53:35,240 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_management.yaml' 2026-05-11T15:53:35,241 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_marketing.yaml' 2026-05-11T15:53:35,242 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_medical_genetics.yaml' 2026-05-11T15:53:35,243 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_miscellaneous.yaml' 2026-05-11T15:53:35,245 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_disputes.yaml' 2026-05-11T15:53:35,246 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_moral_scenarios.yaml' 2026-05-11T15:53:35,247 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_nutrition.yaml' 2026-05-11T15:53:35,248 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_philosophy.yaml' 2026-05-11T15:53:35,249 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_prehistory.yaml' 2026-05-11T15:53:35,250 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_accounting.yaml' 2026-05-11T15:53:35,251 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_law.yaml' 2026-05-11T15:53:35,253 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_medicine.yaml' 2026-05-11T15:53:35,254 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_professional_psychology.yaml' 2026-05-11T15:53:35,255 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_public_relations.yaml' 2026-05-11T15:53:35,256 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_security_studies.yaml' 2026-05-11T15:53:35,257 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_sociology.yaml' 2026-05-11T15:53:35,258 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_us_foreign_policy.yaml' 2026-05-11T15:53:35,260 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_virology.yaml' 2026-05-11T15:53:35,261 adding 'lm_eval/tasks/global_mmlu/full/uk/global_mmlu_full_uk_world_religions.yaml' 2026-05-11T15:53:35,262 adding 'lm_eval/tasks/global_mmlu/full/uk/utils.py' 2026-05-11T15:53:35,265 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi.yaml' 2026-05-11T15:53:35,266 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_humanities.yaml' 2026-05-11T15:53:35,267 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_other.yaml' 2026-05-11T15:53:35,269 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_social_sciences.yaml' 2026-05-11T15:53:35,270 adding 'lm_eval/tasks/global_mmlu/full/vi/_global_mmlu_full_vi_stem.yaml' 2026-05-11T15:53:35,271 adding 'lm_eval/tasks/global_mmlu/full/vi/_vi_template_yaml' 2026-05-11T15:53:35,272 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_abstract_algebra.yaml' 2026-05-11T15:53:35,273 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_anatomy.yaml' 2026-05-11T15:53:35,275 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_astronomy.yaml' 2026-05-11T15:53:35,276 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_business_ethics.yaml' 2026-05-11T15:53:35,277 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_clinical_knowledge.yaml' 2026-05-11T15:53:35,278 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_biology.yaml' 2026-05-11T15:53:35,279 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_chemistry.yaml' 2026-05-11T15:53:35,280 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_computer_science.yaml' 2026-05-11T15:53:35,282 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_mathematics.yaml' 2026-05-11T15:53:35,283 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_medicine.yaml' 2026-05-11T15:53:35,284 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_college_physics.yaml' 2026-05-11T15:53:35,285 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_computer_security.yaml' 2026-05-11T15:53:35,287 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_conceptual_physics.yaml' 2026-05-11T15:53:35,288 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_econometrics.yaml' 2026-05-11T15:53:35,289 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_electrical_engineering.yaml' 2026-05-11T15:53:35,290 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_elementary_mathematics.yaml' 2026-05-11T15:53:35,291 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_formal_logic.yaml' 2026-05-11T15:53:35,292 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_global_facts.yaml' 2026-05-11T15:53:35,293 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_biology.yaml' 2026-05-11T15:53:35,295 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_chemistry.yaml' 2026-05-11T15:53:35,296 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_computer_science.yaml' 2026-05-11T15:53:35,297 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_european_history.yaml' 2026-05-11T15:53:35,298 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_geography.yaml' 2026-05-11T15:53:35,299 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_government_and_politics.yaml' 2026-05-11T15:53:35,300 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_macroeconomics.yaml' 2026-05-11T15:53:35,302 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_mathematics.yaml' 2026-05-11T15:53:35,303 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_microeconomics.yaml' 2026-05-11T15:53:35,304 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_physics.yaml' 2026-05-11T15:53:35,305 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_psychology.yaml' 2026-05-11T15:53:35,306 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_statistics.yaml' 2026-05-11T15:53:35,307 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_us_history.yaml' 2026-05-11T15:53:35,309 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_high_school_world_history.yaml' 2026-05-11T15:53:35,310 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_aging.yaml' 2026-05-11T15:53:35,311 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_human_sexuality.yaml' 2026-05-11T15:53:35,312 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_international_law.yaml' 2026-05-11T15:53:35,313 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_jurisprudence.yaml' 2026-05-11T15:53:35,315 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_logical_fallacies.yaml' 2026-05-11T15:53:35,316 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_machine_learning.yaml' 2026-05-11T15:53:35,317 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_management.yaml' 2026-05-11T15:53:35,318 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_marketing.yaml' 2026-05-11T15:53:35,319 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_medical_genetics.yaml' 2026-05-11T15:53:35,321 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_miscellaneous.yaml' 2026-05-11T15:53:35,322 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_disputes.yaml' 2026-05-11T15:53:35,323 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_moral_scenarios.yaml' 2026-05-11T15:53:35,324 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_nutrition.yaml' 2026-05-11T15:53:35,325 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_philosophy.yaml' 2026-05-11T15:53:35,326 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_prehistory.yaml' 2026-05-11T15:53:35,327 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_accounting.yaml' 2026-05-11T15:53:35,328 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_law.yaml' 2026-05-11T15:53:35,330 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_medicine.yaml' 2026-05-11T15:53:35,331 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_professional_psychology.yaml' 2026-05-11T15:53:35,332 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_public_relations.yaml' 2026-05-11T15:53:35,333 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_security_studies.yaml' 2026-05-11T15:53:35,334 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_sociology.yaml' 2026-05-11T15:53:35,335 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_us_foreign_policy.yaml' 2026-05-11T15:53:35,336 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_virology.yaml' 2026-05-11T15:53:35,337 adding 'lm_eval/tasks/global_mmlu/full/vi/global_mmlu_full_vi_world_religions.yaml' 2026-05-11T15:53:35,339 adding 'lm_eval/tasks/global_mmlu/full/vi/utils.py' 2026-05-11T15:53:35,342 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo.yaml' 2026-05-11T15:53:35,343 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_humanities.yaml' 2026-05-11T15:53:35,344 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_other.yaml' 2026-05-11T15:53:35,346 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_social_sciences.yaml' 2026-05-11T15:53:35,347 adding 'lm_eval/tasks/global_mmlu/full/yo/_global_mmlu_full_yo_stem.yaml' 2026-05-11T15:53:35,348 adding 'lm_eval/tasks/global_mmlu/full/yo/_yo_template_yaml' 2026-05-11T15:53:35,349 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_abstract_algebra.yaml' 2026-05-11T15:53:35,350 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_anatomy.yaml' 2026-05-11T15:53:35,351 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_astronomy.yaml' 2026-05-11T15:53:35,353 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_business_ethics.yaml' 2026-05-11T15:53:35,354 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_clinical_knowledge.yaml' 2026-05-11T15:53:35,355 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_biology.yaml' 2026-05-11T15:53:35,356 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_chemistry.yaml' 2026-05-11T15:53:35,358 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_computer_science.yaml' 2026-05-11T15:53:35,359 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_mathematics.yaml' 2026-05-11T15:53:35,360 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_medicine.yaml' 2026-05-11T15:53:35,361 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_college_physics.yaml' 2026-05-11T15:53:35,362 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_computer_security.yaml' 2026-05-11T15:53:35,363 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_conceptual_physics.yaml' 2026-05-11T15:53:35,365 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_econometrics.yaml' 2026-05-11T15:53:35,366 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_electrical_engineering.yaml' 2026-05-11T15:53:35,367 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_elementary_mathematics.yaml' 2026-05-11T15:53:35,368 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_formal_logic.yaml' 2026-05-11T15:53:35,369 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_global_facts.yaml' 2026-05-11T15:53:35,370 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_biology.yaml' 2026-05-11T15:53:35,371 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_chemistry.yaml' 2026-05-11T15:53:35,372 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_computer_science.yaml' 2026-05-11T15:53:35,374 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_european_history.yaml' 2026-05-11T15:53:35,375 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_geography.yaml' 2026-05-11T15:53:35,376 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_government_and_politics.yaml' 2026-05-11T15:53:35,377 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_macroeconomics.yaml' 2026-05-11T15:53:35,378 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_mathematics.yaml' 2026-05-11T15:53:35,380 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_microeconomics.yaml' 2026-05-11T15:53:35,381 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_physics.yaml' 2026-05-11T15:53:35,382 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_psychology.yaml' 2026-05-11T15:53:35,383 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_statistics.yaml' 2026-05-11T15:53:35,384 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_us_history.yaml' 2026-05-11T15:53:35,385 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_high_school_world_history.yaml' 2026-05-11T15:53:35,386 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_aging.yaml' 2026-05-11T15:53:35,387 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_human_sexuality.yaml' 2026-05-11T15:53:35,389 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_international_law.yaml' 2026-05-11T15:53:35,390 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_jurisprudence.yaml' 2026-05-11T15:53:35,391 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_logical_fallacies.yaml' 2026-05-11T15:53:35,392 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_machine_learning.yaml' 2026-05-11T15:53:35,393 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_management.yaml' 2026-05-11T15:53:35,394 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_marketing.yaml' 2026-05-11T15:53:35,396 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_medical_genetics.yaml' 2026-05-11T15:53:35,397 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_miscellaneous.yaml' 2026-05-11T15:53:35,398 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_disputes.yaml' 2026-05-11T15:53:35,399 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_moral_scenarios.yaml' 2026-05-11T15:53:35,400 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_nutrition.yaml' 2026-05-11T15:53:35,401 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_philosophy.yaml' 2026-05-11T15:53:35,403 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_prehistory.yaml' 2026-05-11T15:53:35,404 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_accounting.yaml' 2026-05-11T15:53:35,405 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_law.yaml' 2026-05-11T15:53:35,407 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_medicine.yaml' 2026-05-11T15:53:35,408 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_professional_psychology.yaml' 2026-05-11T15:53:35,409 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_public_relations.yaml' 2026-05-11T15:53:35,410 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_security_studies.yaml' 2026-05-11T15:53:35,411 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_sociology.yaml' 2026-05-11T15:53:35,412 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_us_foreign_policy.yaml' 2026-05-11T15:53:35,413 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_virology.yaml' 2026-05-11T15:53:35,415 adding 'lm_eval/tasks/global_mmlu/full/yo/global_mmlu_full_yo_world_religions.yaml' 2026-05-11T15:53:35,416 adding 'lm_eval/tasks/global_mmlu/full/yo/utils.py' 2026-05-11T15:53:35,419 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh.yaml' 2026-05-11T15:53:35,420 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_humanities.yaml' 2026-05-11T15:53:35,421 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_other.yaml' 2026-05-11T15:53:35,422 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_social_sciences.yaml' 2026-05-11T15:53:35,424 adding 'lm_eval/tasks/global_mmlu/full/zh/_global_mmlu_full_zh_stem.yaml' 2026-05-11T15:53:35,425 adding 'lm_eval/tasks/global_mmlu/full/zh/_zh_template_yaml' 2026-05-11T15:53:35,426 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_abstract_algebra.yaml' 2026-05-11T15:53:35,427 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_anatomy.yaml' 2026-05-11T15:53:35,428 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_astronomy.yaml' 2026-05-11T15:53:35,429 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_business_ethics.yaml' 2026-05-11T15:53:35,431 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_clinical_knowledge.yaml' 2026-05-11T15:53:35,432 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_biology.yaml' 2026-05-11T15:53:35,433 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_chemistry.yaml' 2026-05-11T15:53:35,434 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_computer_science.yaml' 2026-05-11T15:53:35,435 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_mathematics.yaml' 2026-05-11T15:53:35,436 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_medicine.yaml' 2026-05-11T15:53:35,437 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_college_physics.yaml' 2026-05-11T15:53:35,439 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_computer_security.yaml' 2026-05-11T15:53:35,440 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_conceptual_physics.yaml' 2026-05-11T15:53:35,441 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_econometrics.yaml' 2026-05-11T15:53:35,442 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_electrical_engineering.yaml' 2026-05-11T15:53:35,443 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_elementary_mathematics.yaml' 2026-05-11T15:53:35,444 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_formal_logic.yaml' 2026-05-11T15:53:35,446 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_global_facts.yaml' 2026-05-11T15:53:35,447 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_biology.yaml' 2026-05-11T15:53:35,448 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_chemistry.yaml' 2026-05-11T15:53:35,449 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_computer_science.yaml' 2026-05-11T15:53:35,450 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_european_history.yaml' 2026-05-11T15:53:35,452 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_geography.yaml' 2026-05-11T15:53:35,453 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_government_and_politics.yaml' 2026-05-11T15:53:35,454 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_macroeconomics.yaml' 2026-05-11T15:53:35,455 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_mathematics.yaml' 2026-05-11T15:53:35,456 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_microeconomics.yaml' 2026-05-11T15:53:35,457 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_physics.yaml' 2026-05-11T15:53:35,459 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_psychology.yaml' 2026-05-11T15:53:35,460 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_statistics.yaml' 2026-05-11T15:53:35,461 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_us_history.yaml' 2026-05-11T15:53:35,462 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_high_school_world_history.yaml' 2026-05-11T15:53:35,463 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_aging.yaml' 2026-05-11T15:53:35,464 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_human_sexuality.yaml' 2026-05-11T15:53:35,465 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_international_law.yaml' 2026-05-11T15:53:35,467 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_jurisprudence.yaml' 2026-05-11T15:53:35,468 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_logical_fallacies.yaml' 2026-05-11T15:53:35,469 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_machine_learning.yaml' 2026-05-11T15:53:35,470 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_management.yaml' 2026-05-11T15:53:35,471 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_marketing.yaml' 2026-05-11T15:53:35,472 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_medical_genetics.yaml' 2026-05-11T15:53:35,473 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_miscellaneous.yaml' 2026-05-11T15:53:35,474 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_disputes.yaml' 2026-05-11T15:53:35,475 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_moral_scenarios.yaml' 2026-05-11T15:53:35,477 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_nutrition.yaml' 2026-05-11T15:53:35,478 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_philosophy.yaml' 2026-05-11T15:53:35,479 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_prehistory.yaml' 2026-05-11T15:53:35,480 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_accounting.yaml' 2026-05-11T15:53:35,481 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_law.yaml' 2026-05-11T15:53:35,483 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_medicine.yaml' 2026-05-11T15:53:35,484 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_professional_psychology.yaml' 2026-05-11T15:53:35,485 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_public_relations.yaml' 2026-05-11T15:53:35,486 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_security_studies.yaml' 2026-05-11T15:53:35,487 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_sociology.yaml' 2026-05-11T15:53:35,489 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_us_foreign_policy.yaml' 2026-05-11T15:53:35,490 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_virology.yaml' 2026-05-11T15:53:35,491 adding 'lm_eval/tasks/global_mmlu/full/zh/global_mmlu_full_zh_world_religions.yaml' 2026-05-11T15:53:35,492 adding 'lm_eval/tasks/global_mmlu/full/zh/utils.py' 2026-05-11T15:53:35,494 adding 'lm_eval/tasks/global_piqa/README.md' 2026-05-11T15:53:35,498 adding 'lm_eval/tasks/global_piqa/completions/_generate_config.py' 2026-05-11T15:53:35,500 adding 'lm_eval/tasks/global_piqa/completions/_global_piqa.yaml' 2026-05-11T15:53:35,501 adding 'lm_eval/tasks/global_piqa/completions/_template' 2026-05-11T15:53:35,502 adding 'lm_eval/tasks/global_piqa/completions/acm_arab.yaml' 2026-05-11T15:53:35,503 adding 'lm_eval/tasks/global_piqa/completions/acq_arab.yaml' 2026-05-11T15:53:35,504 adding 'lm_eval/tasks/global_piqa/completions/aeb_arab.yaml' 2026-05-11T15:53:35,506 adding 'lm_eval/tasks/global_piqa/completions/afb_arab.yaml' 2026-05-11T15:53:35,507 adding 'lm_eval/tasks/global_piqa/completions/als_latn.yaml' 2026-05-11T15:53:35,508 adding 'lm_eval/tasks/global_piqa/completions/amh_ethi.yaml' 2026-05-11T15:53:35,509 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_jord.yaml' 2026-05-11T15:53:35,510 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_leba.yaml' 2026-05-11T15:53:35,511 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_pale.yaml' 2026-05-11T15:53:35,512 adding 'lm_eval/tasks/global_piqa/completions/apc_arab_syri.yaml' 2026-05-11T15:53:35,513 adding 'lm_eval/tasks/global_piqa/completions/arb_arab.yaml' 2026-05-11T15:53:35,514 adding 'lm_eval/tasks/global_piqa/completions/arq_arab.yaml' 2026-05-11T15:53:35,515 adding 'lm_eval/tasks/global_piqa/completions/ars_arab.yaml' 2026-05-11T15:53:35,516 adding 'lm_eval/tasks/global_piqa/completions/ary_arab.yaml' 2026-05-11T15:53:35,518 adding 'lm_eval/tasks/global_piqa/completions/arz_arab.yaml' 2026-05-11T15:53:35,519 adding 'lm_eval/tasks/global_piqa/completions/asm_beng.yaml' 2026-05-11T15:53:35,520 adding 'lm_eval/tasks/global_piqa/completions/azj_latn.yaml' 2026-05-11T15:53:35,521 adding 'lm_eval/tasks/global_piqa/completions/bam_latn.yaml' 2026-05-11T15:53:35,522 adding 'lm_eval/tasks/global_piqa/completions/bel_cyrl.yaml' 2026-05-11T15:53:35,523 adding 'lm_eval/tasks/global_piqa/completions/ben_beng.yaml' 2026-05-11T15:53:35,524 adding 'lm_eval/tasks/global_piqa/completions/ben_latn.yaml' 2026-05-11T15:53:35,525 adding 'lm_eval/tasks/global_piqa/completions/bho_deva.yaml' 2026-05-11T15:53:35,526 adding 'lm_eval/tasks/global_piqa/completions/bos_latn.yaml' 2026-05-11T15:53:35,528 adding 'lm_eval/tasks/global_piqa/completions/bsk_arab.yaml' 2026-05-11T15:53:35,529 adding 'lm_eval/tasks/global_piqa/completions/bul_cyrl.yaml' 2026-05-11T15:53:35,530 adding 'lm_eval/tasks/global_piqa/completions/cat_latn.yaml' 2026-05-11T15:53:35,531 adding 'lm_eval/tasks/global_piqa/completions/ces_latn.yaml' 2026-05-11T15:53:35,532 adding 'lm_eval/tasks/global_piqa/completions/ckb_arab.yaml' 2026-05-11T15:53:35,533 adding 'lm_eval/tasks/global_piqa/completions/ckm_latn.yaml' 2026-05-11T15:53:35,534 adding 'lm_eval/tasks/global_piqa/completions/cmn_hans.yaml' 2026-05-11T15:53:35,536 adding 'lm_eval/tasks/global_piqa/completions/cmn_hant.yaml' 2026-05-11T15:53:35,537 adding 'lm_eval/tasks/global_piqa/completions/deu_latn.yaml' 2026-05-11T15:53:35,538 adding 'lm_eval/tasks/global_piqa/completions/dhd_deva.yaml' 2026-05-11T15:53:35,539 adding 'lm_eval/tasks/global_piqa/completions/ekk_latn.yaml' 2026-05-11T15:53:35,540 adding 'lm_eval/tasks/global_piqa/completions/ekp_latn.yaml' 2026-05-11T15:53:35,541 adding 'lm_eval/tasks/global_piqa/completions/ell_grek.yaml' 2026-05-11T15:53:35,542 adding 'lm_eval/tasks/global_piqa/completions/eng_latn.yaml' 2026-05-11T15:53:35,544 adding 'lm_eval/tasks/global_piqa/completions/fao_latn.yaml' 2026-05-11T15:53:35,545 adding 'lm_eval/tasks/global_piqa/completions/fin_latn.yaml' 2026-05-11T15:53:35,546 adding 'lm_eval/tasks/global_piqa/completions/fra_latn_cana.yaml' 2026-05-11T15:53:35,547 adding 'lm_eval/tasks/global_piqa/completions/fra_latn_fran.yaml' 2026-05-11T15:53:35,548 adding 'lm_eval/tasks/global_piqa/completions/glg_latn.yaml' 2026-05-11T15:53:35,549 adding 'lm_eval/tasks/global_piqa/completions/guj_gujr.yaml' 2026-05-11T15:53:35,551 adding 'lm_eval/tasks/global_piqa/completions/hau_latn.yaml' 2026-05-11T15:53:35,552 adding 'lm_eval/tasks/global_piqa/completions/haw_latn.yaml' 2026-05-11T15:53:35,553 adding 'lm_eval/tasks/global_piqa/completions/heb_hebr.yaml' 2026-05-11T15:53:35,554 adding 'lm_eval/tasks/global_piqa/completions/hin_deva.yaml' 2026-05-11T15:53:35,555 adding 'lm_eval/tasks/global_piqa/completions/hrv_latn.yaml' 2026-05-11T15:53:35,556 adding 'lm_eval/tasks/global_piqa/completions/hun_latn.yaml' 2026-05-11T15:53:35,557 adding 'lm_eval/tasks/global_piqa/completions/hye_armn.yaml' 2026-05-11T15:53:35,559 adding 'lm_eval/tasks/global_piqa/completions/ibo_latn.yaml' 2026-05-11T15:53:35,560 adding 'lm_eval/tasks/global_piqa/completions/idu_latn.yaml' 2026-05-11T15:53:35,561 adding 'lm_eval/tasks/global_piqa/completions/ind_latn.yaml' 2026-05-11T15:53:35,562 adding 'lm_eval/tasks/global_piqa/completions/isl_latn.yaml' 2026-05-11T15:53:35,563 adding 'lm_eval/tasks/global_piqa/completions/iso_latn.yaml' 2026-05-11T15:53:35,564 adding 'lm_eval/tasks/global_piqa/completions/ita_latn.yaml' 2026-05-11T15:53:35,565 adding 'lm_eval/tasks/global_piqa/completions/jav_latn.yaml' 2026-05-11T15:53:35,566 adding 'lm_eval/tasks/global_piqa/completions/jpn_jpan.yaml' 2026-05-11T15:53:35,567 adding 'lm_eval/tasks/global_piqa/completions/kan_knda.yaml' 2026-05-11T15:53:35,568 adding 'lm_eval/tasks/global_piqa/completions/kat_geor.yaml' 2026-05-11T15:53:35,569 adding 'lm_eval/tasks/global_piqa/completions/kaz_cyrl.yaml' 2026-05-11T15:53:35,570 adding 'lm_eval/tasks/global_piqa/completions/kin_latn.yaml' 2026-05-11T15:53:35,572 adding 'lm_eval/tasks/global_piqa/completions/kir_cyrl.yaml' 2026-05-11T15:53:35,573 adding 'lm_eval/tasks/global_piqa/completions/kor_hang.yaml' 2026-05-11T15:53:35,574 adding 'lm_eval/tasks/global_piqa/completions/lin_latn.yaml' 2026-05-11T15:53:35,575 adding 'lm_eval/tasks/global_piqa/completions/lit_latn.yaml' 2026-05-11T15:53:35,576 adding 'lm_eval/tasks/global_piqa/completions/luo_latn.yaml' 2026-05-11T15:53:35,577 adding 'lm_eval/tasks/global_piqa/completions/mal_mlym.yaml' 2026-05-11T15:53:35,578 adding 'lm_eval/tasks/global_piqa/completions/mar_deva.yaml' 2026-05-11T15:53:35,579 adding 'lm_eval/tasks/global_piqa/completions/mkd_cyrl.yaml' 2026-05-11T15:53:35,581 adding 'lm_eval/tasks/global_piqa/completions/mni_beng.yaml' 2026-05-11T15:53:35,582 adding 'lm_eval/tasks/global_piqa/completions/mni_mtei.yaml' 2026-05-11T15:53:35,583 adding 'lm_eval/tasks/global_piqa/completions/nag_latn.yaml' 2026-05-11T15:53:35,584 adding 'lm_eval/tasks/global_piqa/completions/nld_latn.yaml' 2026-05-11T15:53:35,585 adding 'lm_eval/tasks/global_piqa/completions/nno_latn.yaml' 2026-05-11T15:53:35,586 adding 'lm_eval/tasks/global_piqa/completions/nob_latn.yaml' 2026-05-11T15:53:35,587 adding 'lm_eval/tasks/global_piqa/completions/npi_deva.yaml' 2026-05-11T15:53:35,589 adding 'lm_eval/tasks/global_piqa/completions/pan_guru.yaml' 2026-05-11T15:53:35,590 adding 'lm_eval/tasks/global_piqa/completions/pcm_latn.yaml' 2026-05-11T15:53:35,591 adding 'lm_eval/tasks/global_piqa/completions/pes_arab.yaml' 2026-05-11T15:53:35,592 adding 'lm_eval/tasks/global_piqa/completions/pol_latn.yaml' 2026-05-11T15:53:35,593 adding 'lm_eval/tasks/global_piqa/completions/por_latn_braz.yaml' 2026-05-11T15:53:35,594 adding 'lm_eval/tasks/global_piqa/completions/por_latn_port.yaml' 2026-05-11T15:53:35,596 adding 'lm_eval/tasks/global_piqa/completions/ron_latn.yaml' 2026-05-11T15:53:35,597 adding 'lm_eval/tasks/global_piqa/completions/rus_cyrl.yaml' 2026-05-11T15:53:35,598 adding 'lm_eval/tasks/global_piqa/completions/rwr_deva.yaml' 2026-05-11T15:53:35,599 adding 'lm_eval/tasks/global_piqa/completions/sin_sinh.yaml' 2026-05-11T15:53:35,600 adding 'lm_eval/tasks/global_piqa/completions/slk_latn.yaml' 2026-05-11T15:53:35,601 adding 'lm_eval/tasks/global_piqa/completions/slk_latn_sari.yaml' 2026-05-11T15:53:35,603 adding 'lm_eval/tasks/global_piqa/completions/slv_latn.yaml' 2026-05-11T15:53:35,604 adding 'lm_eval/tasks/global_piqa/completions/slv_latn_cerk.yaml' 2026-05-11T15:53:35,605 adding 'lm_eval/tasks/global_piqa/completions/snd_arab.yaml' 2026-05-11T15:53:35,606 adding 'lm_eval/tasks/global_piqa/completions/snd_deva.yaml' 2026-05-11T15:53:35,607 adding 'lm_eval/tasks/global_piqa/completions/spa_latn_mexi.yaml' 2026-05-11T15:53:35,609 adding 'lm_eval/tasks/global_piqa/completions/spa_latn_peru.yaml' 2026-05-11T15:53:35,610 adding 'lm_eval/tasks/global_piqa/completions/spa_latn_spai.yaml' 2026-05-11T15:53:35,611 adding 'lm_eval/tasks/global_piqa/completions/srp_cyrl.yaml' 2026-05-11T15:53:35,612 adding 'lm_eval/tasks/global_piqa/completions/srp_latn.yaml' 2026-05-11T15:53:35,613 adding 'lm_eval/tasks/global_piqa/completions/swe_latn.yaml' 2026-05-11T15:53:35,614 adding 'lm_eval/tasks/global_piqa/completions/swh_latn.yaml' 2026-05-11T15:53:35,615 adding 'lm_eval/tasks/global_piqa/completions/tam_taml.yaml' 2026-05-11T15:53:35,616 adding 'lm_eval/tasks/global_piqa/completions/tel_telu.yaml' 2026-05-11T15:53:35,617 adding 'lm_eval/tasks/global_piqa/completions/tgl_latn.yaml' 2026-05-11T15:53:35,618 adding 'lm_eval/tasks/global_piqa/completions/tha_thai.yaml' 2026-05-11T15:53:35,619 adding 'lm_eval/tasks/global_piqa/completions/tur_latn.yaml' 2026-05-11T15:53:35,620 adding 'lm_eval/tasks/global_piqa/completions/uig_arab.yaml' 2026-05-11T15:53:35,621 adding 'lm_eval/tasks/global_piqa/completions/ukr_cyrl.yaml' 2026-05-11T15:53:35,623 adding 'lm_eval/tasks/global_piqa/completions/urd_arab.yaml' 2026-05-11T15:53:35,624 adding 'lm_eval/tasks/global_piqa/completions/urd_latn.yaml' 2026-05-11T15:53:35,625 adding 'lm_eval/tasks/global_piqa/completions/urh_latn.yaml' 2026-05-11T15:53:35,626 adding 'lm_eval/tasks/global_piqa/completions/uzn_latn.yaml' 2026-05-11T15:53:35,627 adding 'lm_eval/tasks/global_piqa/completions/vie_latn.yaml' 2026-05-11T15:53:35,628 adding 'lm_eval/tasks/global_piqa/completions/yor_latn.yaml' 2026-05-11T15:53:35,629 adding 'lm_eval/tasks/global_piqa/completions/yue_hant.yaml' 2026-05-11T15:53:35,630 adding 'lm_eval/tasks/global_piqa/completions/zsm_latn.yaml' 2026-05-11T15:53:35,632 adding 'lm_eval/tasks/global_piqa/completions/zul_latn.yaml' 2026-05-11T15:53:35,636 adding 'lm_eval/tasks/global_piqa/prompted/_generate_config.py' 2026-05-11T15:53:35,637 adding 'lm_eval/tasks/global_piqa/prompted/_global_piqa.yaml' 2026-05-11T15:53:35,639 adding 'lm_eval/tasks/global_piqa/prompted/_template' 2026-05-11T15:53:35,640 adding 'lm_eval/tasks/global_piqa/prompted/acm_arab.yaml' 2026-05-11T15:53:35,641 adding 'lm_eval/tasks/global_piqa/prompted/acq_arab.yaml' 2026-05-11T15:53:35,642 adding 'lm_eval/tasks/global_piqa/prompted/aeb_arab.yaml' 2026-05-11T15:53:35,643 adding 'lm_eval/tasks/global_piqa/prompted/afb_arab.yaml' 2026-05-11T15:53:35,645 adding 'lm_eval/tasks/global_piqa/prompted/als_latn.yaml' 2026-05-11T15:53:35,646 adding 'lm_eval/tasks/global_piqa/prompted/amh_ethi.yaml' 2026-05-11T15:53:35,647 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_jord.yaml' 2026-05-11T15:53:35,648 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_leba.yaml' 2026-05-11T15:53:35,649 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_pale.yaml' 2026-05-11T15:53:35,650 adding 'lm_eval/tasks/global_piqa/prompted/apc_arab_syri.yaml' 2026-05-11T15:53:35,652 adding 'lm_eval/tasks/global_piqa/prompted/arb_arab.yaml' 2026-05-11T15:53:35,653 adding 'lm_eval/tasks/global_piqa/prompted/arq_arab.yaml' 2026-05-11T15:53:35,654 adding 'lm_eval/tasks/global_piqa/prompted/ars_arab.yaml' 2026-05-11T15:53:35,655 adding 'lm_eval/tasks/global_piqa/prompted/ary_arab.yaml' 2026-05-11T15:53:35,656 adding 'lm_eval/tasks/global_piqa/prompted/arz_arab.yaml' 2026-05-11T15:53:35,658 adding 'lm_eval/tasks/global_piqa/prompted/asm_beng.yaml' 2026-05-11T15:53:35,659 adding 'lm_eval/tasks/global_piqa/prompted/azj_latn.yaml' 2026-05-11T15:53:35,660 adding 'lm_eval/tasks/global_piqa/prompted/bam_latn.yaml' 2026-05-11T15:53:35,661 adding 'lm_eval/tasks/global_piqa/prompted/bel_cyrl.yaml' 2026-05-11T15:53:35,662 adding 'lm_eval/tasks/global_piqa/prompted/ben_beng.yaml' 2026-05-11T15:53:35,664 adding 'lm_eval/tasks/global_piqa/prompted/ben_latn.yaml' 2026-05-11T15:53:35,665 adding 'lm_eval/tasks/global_piqa/prompted/bho_deva.yaml' 2026-05-11T15:53:35,666 adding 'lm_eval/tasks/global_piqa/prompted/bos_latn.yaml' 2026-05-11T15:53:35,667 adding 'lm_eval/tasks/global_piqa/prompted/bsk_arab.yaml' 2026-05-11T15:53:35,668 adding 'lm_eval/tasks/global_piqa/prompted/bul_cyrl.yaml' 2026-05-11T15:53:35,669 adding 'lm_eval/tasks/global_piqa/prompted/cat_latn.yaml' 2026-05-11T15:53:35,670 adding 'lm_eval/tasks/global_piqa/prompted/ces_latn.yaml' 2026-05-11T15:53:35,672 adding 'lm_eval/tasks/global_piqa/prompted/ckb_arab.yaml' 2026-05-11T15:53:35,673 adding 'lm_eval/tasks/global_piqa/prompted/ckm_latn.yaml' 2026-05-11T15:53:35,674 adding 'lm_eval/tasks/global_piqa/prompted/cmn_hans.yaml' 2026-05-11T15:53:35,675 adding 'lm_eval/tasks/global_piqa/prompted/cmn_hant.yaml' 2026-05-11T15:53:35,676 adding 'lm_eval/tasks/global_piqa/prompted/deu_latn.yaml' 2026-05-11T15:53:35,677 adding 'lm_eval/tasks/global_piqa/prompted/dhd_deva.yaml' 2026-05-11T15:53:35,678 adding 'lm_eval/tasks/global_piqa/prompted/ekk_latn.yaml' 2026-05-11T15:53:35,679 adding 'lm_eval/tasks/global_piqa/prompted/ekp_latn.yaml' 2026-05-11T15:53:35,681 adding 'lm_eval/tasks/global_piqa/prompted/ell_grek.yaml' 2026-05-11T15:53:35,682 adding 'lm_eval/tasks/global_piqa/prompted/eng_latn.yaml' 2026-05-11T15:53:35,683 adding 'lm_eval/tasks/global_piqa/prompted/fao_latn.yaml' 2026-05-11T15:53:35,684 adding 'lm_eval/tasks/global_piqa/prompted/fin_latn.yaml' 2026-05-11T15:53:35,685 adding 'lm_eval/tasks/global_piqa/prompted/fra_latn_cana.yaml' 2026-05-11T15:53:35,686 adding 'lm_eval/tasks/global_piqa/prompted/fra_latn_fran.yaml' 2026-05-11T15:53:35,687 adding 'lm_eval/tasks/global_piqa/prompted/glg_latn.yaml' 2026-05-11T15:53:35,689 adding 'lm_eval/tasks/global_piqa/prompted/guj_gujr.yaml' 2026-05-11T15:53:35,690 adding 'lm_eval/tasks/global_piqa/prompted/hau_latn.yaml' 2026-05-11T15:53:35,691 adding 'lm_eval/tasks/global_piqa/prompted/haw_latn.yaml' 2026-05-11T15:53:35,692 adding 'lm_eval/tasks/global_piqa/prompted/heb_hebr.yaml' 2026-05-11T15:53:35,694 adding 'lm_eval/tasks/global_piqa/prompted/hin_deva.yaml' 2026-05-11T15:53:35,695 adding 'lm_eval/tasks/global_piqa/prompted/hrv_latn.yaml' 2026-05-11T15:53:35,696 adding 'lm_eval/tasks/global_piqa/prompted/hun_latn.yaml' 2026-05-11T15:53:35,697 adding 'lm_eval/tasks/global_piqa/prompted/hye_armn.yaml' 2026-05-11T15:53:35,698 adding 'lm_eval/tasks/global_piqa/prompted/ibo_latn.yaml' 2026-05-11T15:53:35,699 adding 'lm_eval/tasks/global_piqa/prompted/idu_latn.yaml' 2026-05-11T15:53:35,701 adding 'lm_eval/tasks/global_piqa/prompted/ind_latn.yaml' 2026-05-11T15:53:35,702 adding 'lm_eval/tasks/global_piqa/prompted/isl_latn.yaml' 2026-05-11T15:53:35,703 adding 'lm_eval/tasks/global_piqa/prompted/iso_latn.yaml' 2026-05-11T15:53:35,704 adding 'lm_eval/tasks/global_piqa/prompted/ita_latn.yaml' 2026-05-11T15:53:35,705 adding 'lm_eval/tasks/global_piqa/prompted/jav_latn.yaml' 2026-05-11T15:53:35,707 adding 'lm_eval/tasks/global_piqa/prompted/jpn_jpan.yaml' 2026-05-11T15:53:35,708 adding 'lm_eval/tasks/global_piqa/prompted/kan_knda.yaml' 2026-05-11T15:53:35,709 adding 'lm_eval/tasks/global_piqa/prompted/kat_geor.yaml' 2026-05-11T15:53:35,710 adding 'lm_eval/tasks/global_piqa/prompted/kaz_cyrl.yaml' 2026-05-11T15:53:35,711 adding 'lm_eval/tasks/global_piqa/prompted/kin_latn.yaml' 2026-05-11T15:53:35,713 adding 'lm_eval/tasks/global_piqa/prompted/kir_cyrl.yaml' 2026-05-11T15:53:35,714 adding 'lm_eval/tasks/global_piqa/prompted/kor_hang.yaml' 2026-05-11T15:53:35,715 adding 'lm_eval/tasks/global_piqa/prompted/lin_latn.yaml' 2026-05-11T15:53:35,716 adding 'lm_eval/tasks/global_piqa/prompted/lit_latn.yaml' 2026-05-11T15:53:35,717 adding 'lm_eval/tasks/global_piqa/prompted/luo_latn.yaml' 2026-05-11T15:53:35,719 adding 'lm_eval/tasks/global_piqa/prompted/mal_mlym.yaml' 2026-05-11T15:53:35,720 adding 'lm_eval/tasks/global_piqa/prompted/mar_deva.yaml' 2026-05-11T15:53:35,721 adding 'lm_eval/tasks/global_piqa/prompted/mkd_cyrl.yaml' 2026-05-11T15:53:35,722 adding 'lm_eval/tasks/global_piqa/prompted/mni_beng.yaml' 2026-05-11T15:53:35,723 adding 'lm_eval/tasks/global_piqa/prompted/mni_mtei.yaml' 2026-05-11T15:53:35,724 adding 'lm_eval/tasks/global_piqa/prompted/nag_latn.yaml' 2026-05-11T15:53:35,725 adding 'lm_eval/tasks/global_piqa/prompted/nld_latn.yaml' 2026-05-11T15:53:35,726 adding 'lm_eval/tasks/global_piqa/prompted/nno_latn.yaml' 2026-05-11T15:53:35,727 adding 'lm_eval/tasks/global_piqa/prompted/nob_latn.yaml' 2026-05-11T15:53:35,728 adding 'lm_eval/tasks/global_piqa/prompted/npi_deva.yaml' 2026-05-11T15:53:35,729 adding 'lm_eval/tasks/global_piqa/prompted/pan_guru.yaml' 2026-05-11T15:53:35,730 adding 'lm_eval/tasks/global_piqa/prompted/pcm_latn.yaml' 2026-05-11T15:53:35,732 adding 'lm_eval/tasks/global_piqa/prompted/pes_arab.yaml' 2026-05-11T15:53:35,733 adding 'lm_eval/tasks/global_piqa/prompted/pol_latn.yaml' 2026-05-11T15:53:35,734 adding 'lm_eval/tasks/global_piqa/prompted/por_latn_braz.yaml' 2026-05-11T15:53:35,735 adding 'lm_eval/tasks/global_piqa/prompted/por_latn_port.yaml' 2026-05-11T15:53:35,736 adding 'lm_eval/tasks/global_piqa/prompted/ron_latn.yaml' 2026-05-11T15:53:35,737 adding 'lm_eval/tasks/global_piqa/prompted/rus_cyrl.yaml' 2026-05-11T15:53:35,738 adding 'lm_eval/tasks/global_piqa/prompted/rwr_deva.yaml' 2026-05-11T15:53:35,739 adding 'lm_eval/tasks/global_piqa/prompted/sin_sinh.yaml' 2026-05-11T15:53:35,741 adding 'lm_eval/tasks/global_piqa/prompted/slk_latn.yaml' 2026-05-11T15:53:35,742 adding 'lm_eval/tasks/global_piqa/prompted/slk_latn_sari.yaml' 2026-05-11T15:53:35,743 adding 'lm_eval/tasks/global_piqa/prompted/slv_latn.yaml' 2026-05-11T15:53:35,744 adding 'lm_eval/tasks/global_piqa/prompted/slv_latn_cerk.yaml' 2026-05-11T15:53:35,745 adding 'lm_eval/tasks/global_piqa/prompted/snd_arab.yaml' 2026-05-11T15:53:35,746 adding 'lm_eval/tasks/global_piqa/prompted/snd_deva.yaml' 2026-05-11T15:53:35,747 adding 'lm_eval/tasks/global_piqa/prompted/spa_latn_mexi.yaml' 2026-05-11T15:53:35,749 adding 'lm_eval/tasks/global_piqa/prompted/spa_latn_peru.yaml' 2026-05-11T15:53:35,750 adding 'lm_eval/tasks/global_piqa/prompted/spa_latn_spai.yaml' 2026-05-11T15:53:35,751 adding 'lm_eval/tasks/global_piqa/prompted/srp_cyrl.yaml' 2026-05-11T15:53:35,752 adding 'lm_eval/tasks/global_piqa/prompted/srp_latn.yaml' 2026-05-11T15:53:35,753 adding 'lm_eval/tasks/global_piqa/prompted/swe_latn.yaml' 2026-05-11T15:53:35,755 adding 'lm_eval/tasks/global_piqa/prompted/swh_latn.yaml' 2026-05-11T15:53:35,756 adding 'lm_eval/tasks/global_piqa/prompted/tam_taml.yaml' 2026-05-11T15:53:35,757 adding 'lm_eval/tasks/global_piqa/prompted/tel_telu.yaml' 2026-05-11T15:53:35,758 adding 'lm_eval/tasks/global_piqa/prompted/tgl_latn.yaml' 2026-05-11T15:53:35,759 adding 'lm_eval/tasks/global_piqa/prompted/tha_thai.yaml' 2026-05-11T15:53:35,760 adding 'lm_eval/tasks/global_piqa/prompted/tur_latn.yaml' 2026-05-11T15:53:35,761 adding 'lm_eval/tasks/global_piqa/prompted/uig_arab.yaml' 2026-05-11T15:53:35,762 adding 'lm_eval/tasks/global_piqa/prompted/ukr_cyrl.yaml' 2026-05-11T15:53:35,763 adding 'lm_eval/tasks/global_piqa/prompted/urd_arab.yaml' 2026-05-11T15:53:35,765 adding 'lm_eval/tasks/global_piqa/prompted/urd_latn.yaml' 2026-05-11T15:53:35,766 adding 'lm_eval/tasks/global_piqa/prompted/urh_latn.yaml' 2026-05-11T15:53:35,767 adding 'lm_eval/tasks/global_piqa/prompted/uzn_latn.yaml' 2026-05-11T15:53:35,768 adding 'lm_eval/tasks/global_piqa/prompted/vie_latn.yaml' 2026-05-11T15:53:35,770 adding 'lm_eval/tasks/global_piqa/prompted/yor_latn.yaml' 2026-05-11T15:53:35,771 adding 'lm_eval/tasks/global_piqa/prompted/yue_hant.yaml' 2026-05-11T15:53:35,772 adding 'lm_eval/tasks/global_piqa/prompted/zsm_latn.yaml' 2026-05-11T15:53:35,773 adding 'lm_eval/tasks/global_piqa/prompted/zul_latn.yaml' 2026-05-11T15:53:35,775 adding 'lm_eval/tasks/glue/README.md' 2026-05-11T15:53:35,777 adding 'lm_eval/tasks/glue/cola/default.yaml' 2026-05-11T15:53:35,779 adding 'lm_eval/tasks/glue/mnli/default.yaml' 2026-05-11T15:53:35,780 adding 'lm_eval/tasks/glue/mnli/mismatch.yaml' 2026-05-11T15:53:35,781 adding 'lm_eval/tasks/glue/mnli/utils.py' 2026-05-11T15:53:35,783 adding 'lm_eval/tasks/glue/mrpc/default.yaml' 2026-05-11T15:53:35,784 adding 'lm_eval/tasks/glue/qnli/default.yaml' 2026-05-11T15:53:35,786 adding 'lm_eval/tasks/glue/qqp/default.yaml' 2026-05-11T15:53:35,787 adding 'lm_eval/tasks/glue/rte/default.yaml' 2026-05-11T15:53:35,789 adding 'lm_eval/tasks/glue/sst2/default.yaml' 2026-05-11T15:53:35,790 adding 'lm_eval/tasks/glue/wnli/default.yaml' 2026-05-11T15:53:35,792 adding 'lm_eval/tasks/gpqa/README.md' 2026-05-11T15:53:35,794 adding 'lm_eval/tasks/gpqa/cot_n_shot/_generate_configs.py' 2026-05-11T15:53:35,795 adding 'lm_eval/tasks/gpqa/cot_n_shot/_gpqa_cot_n_shot_yaml' 2026-05-11T15:53:35,796 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_diamond_cot_n_shot.yaml' 2026-05-11T15:53:35,797 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_extended_cot_n_shot.yaml' 2026-05-11T15:53:35,798 adding 'lm_eval/tasks/gpqa/cot_n_shot/gpqa_main_cot_n_shot.yaml' 2026-05-11T15:53:35,800 adding 'lm_eval/tasks/gpqa/cot_n_shot/utils.py' 2026-05-11T15:53:35,801 adding 'lm_eval/tasks/gpqa/cot_zeroshot/_generate_configs.py' 2026-05-11T15:53:35,803 adding 'lm_eval/tasks/gpqa/cot_zeroshot/_gpqa_cot_zeroshot_yaml' 2026-05-11T15:53:35,804 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_diamond_cot_zeroshot.yaml' 2026-05-11T15:53:35,805 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_extended_cot_zeroshot.yaml' 2026-05-11T15:53:35,806 adding 'lm_eval/tasks/gpqa/cot_zeroshot/gpqa_main_cot_zeroshot.yaml' 2026-05-11T15:53:35,807 adding 'lm_eval/tasks/gpqa/cot_zeroshot/utils.py' 2026-05-11T15:53:35,809 adding 'lm_eval/tasks/gpqa/generative/_generate_configs.py' 2026-05-11T15:53:35,810 adding 'lm_eval/tasks/gpqa/generative/_gpqa_generative_n_shot_yaml' 2026-05-11T15:53:35,811 adding 'lm_eval/tasks/gpqa/generative/gpqa_diamond_generative_n_shot.yaml' 2026-05-11T15:53:35,813 adding 'lm_eval/tasks/gpqa/generative/gpqa_extended_generative_n_shot.yaml' 2026-05-11T15:53:35,814 adding 'lm_eval/tasks/gpqa/generative/gpqa_main_generative_n_shot.yaml' 2026-05-11T15:53:35,815 adding 'lm_eval/tasks/gpqa/generative/utils.py' 2026-05-11T15:53:35,817 adding 'lm_eval/tasks/gpqa/n_shot/_generate_configs.py' 2026-05-11T15:53:35,818 adding 'lm_eval/tasks/gpqa/n_shot/_gpqa_n_shot_yaml' 2026-05-11T15:53:35,819 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_diamond_n_shot.yaml' 2026-05-11T15:53:35,820 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_extended_n_shot.yaml' 2026-05-11T15:53:35,821 adding 'lm_eval/tasks/gpqa/n_shot/gpqa_main_n_shot.yaml' 2026-05-11T15:53:35,823 adding 'lm_eval/tasks/gpqa/n_shot/utils.py' 2026-05-11T15:53:35,824 adding 'lm_eval/tasks/gpqa/zeroshot/_generate_configs.py' 2026-05-11T15:53:35,826 adding 'lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml' 2026-05-11T15:53:35,827 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_diamond_zeroshot.yaml' 2026-05-11T15:53:35,828 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_extended_zeroshot.yaml' 2026-05-11T15:53:35,829 adding 'lm_eval/tasks/gpqa/zeroshot/gpqa_main_zeroshot.yaml' 2026-05-11T15:53:35,830 adding 'lm_eval/tasks/gpqa/zeroshot/utils.py' 2026-05-11T15:53:35,832 adding 'lm_eval/tasks/graphwalks/README.md' 2026-05-11T15:53:35,834 adding 'lm_eval/tasks/graphwalks/graphwalks.yaml' 2026-05-11T15:53:35,835 adding 'lm_eval/tasks/graphwalks/graphwalks_128k.yaml' 2026-05-11T15:53:35,836 adding 'lm_eval/tasks/graphwalks/graphwalks_1M.yaml' 2026-05-11T15:53:35,837 adding 'lm_eval/tasks/graphwalks/utils.py' 2026-05-11T15:53:35,839 adding 'lm_eval/tasks/groundcocoa/README.md' 2026-05-11T15:53:35,841 adding 'lm_eval/tasks/groundcocoa/groundcocoa.yaml' 2026-05-11T15:53:35,842 adding 'lm_eval/tasks/groundcocoa/utils.py' 2026-05-11T15:53:35,844 adding 'lm_eval/tasks/gsm8k/README.md' 2026-05-11T15:53:35,845 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-llama.yaml' 2026-05-11T15:53:35,847 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-self-consistency.yaml' 2026-05-11T15:53:35,848 adding 'lm_eval/tasks/gsm8k/gsm8k-cot-zeroshot.yaml' 2026-05-11T15:53:35,849 adding 'lm_eval/tasks/gsm8k/gsm8k-cot.yaml' 2026-05-11T15:53:35,851 adding 'lm_eval/tasks/gsm8k/gsm8k.yaml' 2026-05-11T15:53:35,853 adding 'lm_eval/tasks/gsm8k_platinum/README.md' 2026-05-11T15:53:35,854 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-llama.yaml' 2026-05-11T15:53:35,855 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-self-consistency.yaml' 2026-05-11T15:53:35,856 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot-zeroshot.yaml' 2026-05-11T15:53:35,858 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum-cot.yaml' 2026-05-11T15:53:35,859 adding 'lm_eval/tasks/gsm8k_platinum/gsm8k-platinum.yaml' 2026-05-11T15:53:35,861 adding 'lm_eval/tasks/gsm_plus/README.md' 2026-05-11T15:53:35,862 adding 'lm_eval/tasks/gsm_plus/gsm_plus.yaml' 2026-05-11T15:53:35,863 adding 'lm_eval/tasks/gsm_plus/gsm_plus_mini.yaml' 2026-05-11T15:53:35,865 adding 'lm_eval/tasks/haerae/README.md' 2026-05-11T15:53:35,866 adding 'lm_eval/tasks/haerae/_default_haerae_yaml' 2026-05-11T15:53:35,868 adding 'lm_eval/tasks/haerae/_haerae.yaml' 2026-05-11T15:53:35,869 adding 'lm_eval/tasks/haerae/haerae_gk.yaml' 2026-05-11T15:53:35,870 adding 'lm_eval/tasks/haerae/haerae_hi.yaml' 2026-05-11T15:53:35,871 adding 'lm_eval/tasks/haerae/haerae_lw.yaml' 2026-05-11T15:53:35,872 adding 'lm_eval/tasks/haerae/haerae_rw.yaml' 2026-05-11T15:53:35,873 adding 'lm_eval/tasks/haerae/haerae_sn.yaml' 2026-05-11T15:53:35,875 adding 'lm_eval/tasks/headqa/README.md' 2026-05-11T15:53:35,876 adding 'lm_eval/tasks/headqa/headqa_en.yaml' 2026-05-11T15:53:35,878 adding 'lm_eval/tasks/headqa/headqa_es.yaml' 2026-05-11T15:53:35,880 adding 'lm_eval/tasks/hellaswag/README.md' 2026-05-11T15:53:35,881 adding 'lm_eval/tasks/hellaswag/hellaswag.yaml' 2026-05-11T15:53:35,882 adding 'lm_eval/tasks/hellaswag/utils.py' 2026-05-11T15:53:35,884 adding 'lm_eval/tasks/hendrycks_ethics/README.md' 2026-05-11T15:53:35,885 adding 'lm_eval/tasks/hendrycks_ethics/commonsense.yaml' 2026-05-11T15:53:35,886 adding 'lm_eval/tasks/hendrycks_ethics/deontology.yaml' 2026-05-11T15:53:35,887 adding 'lm_eval/tasks/hendrycks_ethics/justice.yaml' 2026-05-11T15:53:35,889 adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism.yaml' 2026-05-11T15:53:35,890 adding 'lm_eval/tasks/hendrycks_ethics/utilitarianism_original_yaml' 2026-05-11T15:53:35,891 adding 'lm_eval/tasks/hendrycks_ethics/utils.py' 2026-05-11T15:53:35,892 adding 'lm_eval/tasks/hendrycks_ethics/virtue.yaml' 2026-05-11T15:53:35,894 adding 'lm_eval/tasks/hendrycks_math/README.md' 2026-05-11T15:53:35,895 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math.yaml' 2026-05-11T15:53:35,897 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math500.yaml' 2026-05-11T15:53:35,898 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_algebra.yaml' 2026-05-11T15:53:35,899 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_counting_and_prob.yaml' 2026-05-11T15:53:35,900 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_geometry.yaml' 2026-05-11T15:53:35,901 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_intermediate_algebra.yaml' 2026-05-11T15:53:35,902 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_num_theory.yaml' 2026-05-11T15:53:35,904 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_prealgebra.yaml' 2026-05-11T15:53:35,905 adding 'lm_eval/tasks/hendrycks_math/hendrycks_math_precalc.yaml' 2026-05-11T15:53:35,906 adding 'lm_eval/tasks/hendrycks_math/utils.py' 2026-05-11T15:53:35,908 adding 'lm_eval/tasks/histoires_morales/README.md' 2026-05-11T15:53:35,910 adding 'lm_eval/tasks/histoires_morales/histoires_morales.yaml' 2026-05-11T15:53:35,911 adding 'lm_eval/tasks/histoires_morales/utils.py' 2026-05-11T15:53:35,913 adding 'lm_eval/tasks/hrm8k/README.md' 2026-05-11T15:53:35,915 adding 'lm_eval/tasks/hrm8k/default/_hrm8k_yaml' 2026-05-11T15:53:35,916 adding 'lm_eval/tasks/hrm8k/default/hrm8k.yaml' 2026-05-11T15:53:35,917 adding 'lm_eval/tasks/hrm8k/default/hrm8k_gsm8k.yaml' 2026-05-11T15:53:35,918 adding 'lm_eval/tasks/hrm8k/default/hrm8k_ksm.yaml' 2026-05-11T15:53:35,919 adding 'lm_eval/tasks/hrm8k/default/hrm8k_math.yaml' 2026-05-11T15:53:35,920 adding 'lm_eval/tasks/hrm8k/default/hrm8k_mmmlu.yaml' 2026-05-11T15:53:35,921 adding 'lm_eval/tasks/hrm8k/default/hrm8k_omni_math.yaml' 2026-05-11T15:53:35,923 adding 'lm_eval/tasks/hrm8k/default/utils.py' 2026-05-11T15:53:35,925 adding 'lm_eval/tasks/hrm8k/en/_hrm8k_en_yaml' 2026-05-11T15:53:35,926 adding 'lm_eval/tasks/hrm8k/en/hrm8k_en.yaml' 2026-05-11T15:53:35,927 adding 'lm_eval/tasks/hrm8k/en/hrm8k_gsm8k_en.yaml' 2026-05-11T15:53:35,928 adding 'lm_eval/tasks/hrm8k/en/hrm8k_ksm_en.yaml' 2026-05-11T15:53:35,929 adding 'lm_eval/tasks/hrm8k/en/hrm8k_math_en.yaml' 2026-05-11T15:53:35,930 adding 'lm_eval/tasks/hrm8k/en/hrm8k_mmmlu_en.yaml' 2026-05-11T15:53:35,931 adding 'lm_eval/tasks/hrm8k/en/hrm8k_omni_math_en.yaml' 2026-05-11T15:53:35,933 adding 'lm_eval/tasks/hrm8k/en/utils.py' 2026-05-11T15:53:35,935 adding 'lm_eval/tasks/humaneval/README.md' 2026-05-11T15:53:35,936 adding 'lm_eval/tasks/humaneval/humaneval.yaml' 2026-05-11T15:53:35,938 adding 'lm_eval/tasks/humaneval/humaneval_64.yaml' 2026-05-11T15:53:35,939 adding 'lm_eval/tasks/humaneval/humaneval_64_instruct.yaml' 2026-05-11T15:53:35,940 adding 'lm_eval/tasks/humaneval/humaneval_instruct.yaml' 2026-05-11T15:53:35,941 adding 'lm_eval/tasks/humaneval/humaneval_plus.yaml' 2026-05-11T15:53:35,942 adding 'lm_eval/tasks/humaneval/utils.py' 2026-05-11T15:53:35,944 adding 'lm_eval/tasks/humaneval_infilling/README.md' 2026-05-11T15:53:35,945 adding 'lm_eval/tasks/humaneval_infilling/humaneval_infilling.yaml' 2026-05-11T15:53:35,947 adding 'lm_eval/tasks/humaneval_infilling/multi_line_infilling.yaml' 2026-05-11T15:53:35,948 adding 'lm_eval/tasks/humaneval_infilling/random_span_infilling.yaml' 2026-05-11T15:53:35,949 adding 'lm_eval/tasks/humaneval_infilling/random_span_infilling_light.yaml' 2026-05-11T15:53:35,950 adding 'lm_eval/tasks/humaneval_infilling/single_line_infilling.yaml' 2026-05-11T15:53:35,951 adding 'lm_eval/tasks/humaneval_infilling/utils.py' 2026-05-11T15:53:35,953 adding 'lm_eval/tasks/icelandic_winogrande/README.md' 2026-05-11T15:53:35,954 adding 'lm_eval/tasks/icelandic_winogrande/default.yaml' 2026-05-11T15:53:35,955 adding 'lm_eval/tasks/icelandic_winogrande/preprocess_winogrande.py' 2026-05-11T15:53:35,957 adding 'lm_eval/tasks/ifeval/README.md' 2026-05-11T15:53:35,958 adding 'lm_eval/tasks/ifeval/ifeval.yaml' 2026-05-11T15:53:35,964 adding 'lm_eval/tasks/ifeval/instructions.py' 2026-05-11T15:53:35,966 adding 'lm_eval/tasks/ifeval/instructions_registry.py' 2026-05-11T15:53:35,969 adding 'lm_eval/tasks/ifeval/instructions_util.py' 2026-05-11T15:53:35,971 adding 'lm_eval/tasks/ifeval/utils.py' 2026-05-11T15:53:35,972 adding 'lm_eval/tasks/ifeval/multilingual/ifeval_ca.yaml' 2026-05-11T15:53:35,974 adding 'lm_eval/tasks/ifeval/multilingual/ifeval_es.yaml' 2026-05-11T15:53:35,975 adding 'lm_eval/tasks/ifeval/multilingual/instructions_registry.py' 2026-05-11T15:53:35,976 adding 'lm_eval/tasks/ifeval/multilingual/utils.py' 2026-05-11T15:53:35,978 adding 'lm_eval/tasks/ifeval/multilingual/instruction_utils/__init__.py' 2026-05-11T15:53:35,980 adding 'lm_eval/tasks/ifeval/multilingual/instruction_utils/ca_instructions_util.py' 2026-05-11T15:53:35,981 adding 'lm_eval/tasks/ifeval/multilingual/instruction_utils/es_instructions_util.py' 2026-05-11T15:53:35,983 adding 'lm_eval/tasks/ifeval/multilingual/instructions/__init__.py' 2026-05-11T15:53:35,989 adding 'lm_eval/tasks/ifeval/multilingual/instructions/ca_instructions.py' 2026-05-11T15:53:35,996 adding 'lm_eval/tasks/ifeval/multilingual/instructions/es_instructions.py' 2026-05-11T15:53:35,998 adding 'lm_eval/tasks/include/README.md' 2026-05-11T15:53:36,001 adding 'lm_eval/tasks/include/default/Albanian/_albanian_template_yaml' 2026-05-11T15:53:36,002 adding 'lm_eval/tasks/include/default/Albanian/_include_base_44_albanian.yaml' 2026-05-11T15:53:36,004 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_arts_humanities.yaml' 2026-05-11T15:53:36,005 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_business_commerce.yaml' 2026-05-11T15:53:36,006 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_health_oriented_education.yaml' 2026-05-11T15:53:36,007 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_social_science.yaml' 2026-05-11T15:53:36,008 adding 'lm_eval/tasks/include/default/Albanian/include_base_44_albanian_stem.yaml' 2026-05-11T15:53:36,009 adding 'lm_eval/tasks/include/default/Albanian/utils.py' 2026-05-11T15:53:36,011 adding 'lm_eval/tasks/include/default/Arabic/_arabic_template_yaml' 2026-05-11T15:53:36,012 adding 'lm_eval/tasks/include/default/Arabic/_include_base_44_arabic.yaml' 2026-05-11T15:53:36,013 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_arts_humanities.yaml' 2026-05-11T15:53:36,014 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_business_commerce.yaml' 2026-05-11T15:53:36,016 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_driving_license.yaml' 2026-05-11T15:53:36,017 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_general_knowledge.yaml' 2026-05-11T15:53:36,018 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_social_science.yaml' 2026-05-11T15:53:36,019 adding 'lm_eval/tasks/include/default/Arabic/include_base_44_arabic_stem.yaml' 2026-05-11T15:53:36,020 adding 'lm_eval/tasks/include/default/Arabic/utils.py' 2026-05-11T15:53:36,022 adding 'lm_eval/tasks/include/default/Armenian/_armenian_template_yaml' 2026-05-11T15:53:36,023 adding 'lm_eval/tasks/include/default/Armenian/_include_base_44_armenian.yaml' 2026-05-11T15:53:36,024 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_arts_humanities.yaml' 2026-05-11T15:53:36,026 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_driving_license.yaml' 2026-05-11T15:53:36,027 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_social_science.yaml' 2026-05-11T15:53:36,028 adding 'lm_eval/tasks/include/default/Armenian/include_base_44_armenian_stem.yaml' 2026-05-11T15:53:36,030 adding 'lm_eval/tasks/include/default/Armenian/utils.py' 2026-05-11T15:53:36,032 adding 'lm_eval/tasks/include/default/Azerbaijani/_azerbaijani_template_yaml' 2026-05-11T15:53:36,033 adding 'lm_eval/tasks/include/default/Azerbaijani/_include_base_44_azerbaijani.yaml' 2026-05-11T15:53:36,034 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml' 2026-05-11T15:53:36,035 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml' 2026-05-11T15:53:36,036 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml' 2026-05-11T15:53:36,038 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml' 2026-05-11T15:53:36,039 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_social_science.yaml' 2026-05-11T15:53:36,040 adding 'lm_eval/tasks/include/default/Azerbaijani/include_base_44_azerbaijani_stem.yaml' 2026-05-11T15:53:36,041 adding 'lm_eval/tasks/include/default/Azerbaijani/utils.py' 2026-05-11T15:53:36,043 adding 'lm_eval/tasks/include/default/Basque/_basque_template_yaml' 2026-05-11T15:53:36,044 adding 'lm_eval/tasks/include/default/Basque/_include_base_44_basque.yaml' 2026-05-11T15:53:36,045 adding 'lm_eval/tasks/include/default/Basque/include_base_44_basque_professional_certification.yaml' 2026-05-11T15:53:36,046 adding 'lm_eval/tasks/include/default/Basque/utils.py' 2026-05-11T15:53:36,048 adding 'lm_eval/tasks/include/default/Belarusian/_belarusian_template_yaml' 2026-05-11T15:53:36,049 adding 'lm_eval/tasks/include/default/Belarusian/_include_base_44_belarusian.yaml' 2026-05-11T15:53:36,051 adding 'lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_arts_humanities.yaml' 2026-05-11T15:53:36,052 adding 'lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_social_science.yaml' 2026-05-11T15:53:36,053 adding 'lm_eval/tasks/include/default/Belarusian/include_base_44_belarusian_stem.yaml' 2026-05-11T15:53:36,054 adding 'lm_eval/tasks/include/default/Belarusian/utils.py' 2026-05-11T15:53:36,055 adding 'lm_eval/tasks/include/default/Bengali/_bengali_template_yaml' 2026-05-11T15:53:36,056 adding 'lm_eval/tasks/include/default/Bengali/_include_base_44_bengali.yaml' 2026-05-11T15:53:36,057 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_arts_humanities.yaml' 2026-05-11T15:53:36,058 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_general_knowledge.yaml' 2026-05-11T15:53:36,060 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_professional_certification.yaml' 2026-05-11T15:53:36,061 adding 'lm_eval/tasks/include/default/Bengali/include_base_44_bengali_stem.yaml' 2026-05-11T15:53:36,062 adding 'lm_eval/tasks/include/default/Bengali/utils.py' 2026-05-11T15:53:36,063 adding 'lm_eval/tasks/include/default/Bulgarian/_bulgarian_template_yaml' 2026-05-11T15:53:36,065 adding 'lm_eval/tasks/include/default/Bulgarian/_include_base_44_bulgarian.yaml' 2026-05-11T15:53:36,066 adding 'lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml' 2026-05-11T15:53:36,067 adding 'lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_social_science.yaml' 2026-05-11T15:53:36,068 adding 'lm_eval/tasks/include/default/Bulgarian/include_base_44_bulgarian_stem.yaml' 2026-05-11T15:53:36,069 adding 'lm_eval/tasks/include/default/Bulgarian/utils.py' 2026-05-11T15:53:36,071 adding 'lm_eval/tasks/include/default/Chinese/_chinese_template_yaml' 2026-05-11T15:53:36,073 adding 'lm_eval/tasks/include/default/Chinese/_include_base_44_chinese.yaml' 2026-05-11T15:53:36,074 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_applied_science.yaml' 2026-05-11T15:53:36,075 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_arts_humanities.yaml' 2026-05-11T15:53:36,076 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_business_commerce.yaml' 2026-05-11T15:53:36,077 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_driving_license.yaml' 2026-05-11T15:53:36,079 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_health_oriented_education.yaml' 2026-05-11T15:53:36,080 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_professional_certification.yaml' 2026-05-11T15:53:36,081 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_social_science.yaml' 2026-05-11T15:53:36,082 adding 'lm_eval/tasks/include/default/Chinese/include_base_44_chinese_stem.yaml' 2026-05-11T15:53:36,083 adding 'lm_eval/tasks/include/default/Chinese/utils.py' 2026-05-11T15:53:36,085 adding 'lm_eval/tasks/include/default/Croatian/_croatian_template_yaml' 2026-05-11T15:53:36,086 adding 'lm_eval/tasks/include/default/Croatian/_include_base_44_croatian.yaml' 2026-05-11T15:53:36,087 adding 'lm_eval/tasks/include/default/Croatian/include_base_44_croatian_arts_humanities.yaml' 2026-05-11T15:53:36,089 adding 'lm_eval/tasks/include/default/Croatian/include_base_44_croatian_social_science.yaml' 2026-05-11T15:53:36,090 adding 'lm_eval/tasks/include/default/Croatian/include_base_44_croatian_stem.yaml' 2026-05-11T15:53:36,091 adding 'lm_eval/tasks/include/default/Croatian/utils.py' 2026-05-11T15:53:36,093 adding 'lm_eval/tasks/include/default/Dutch/_dutch_template_yaml' 2026-05-11T15:53:36,094 adding 'lm_eval/tasks/include/default/Dutch/_include_base_44_dutch.yaml' 2026-05-11T15:53:36,095 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_applied_science.yaml' 2026-05-11T15:53:36,097 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_arts_humanities.yaml' 2026-05-11T15:53:36,098 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_health_oriented_education.yaml' 2026-05-11T15:53:36,099 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_social_science.yaml' 2026-05-11T15:53:36,100 adding 'lm_eval/tasks/include/default/Dutch/include_base_44_dutch_stem.yaml' 2026-05-11T15:53:36,101 adding 'lm_eval/tasks/include/default/Dutch/utils.py' 2026-05-11T15:53:36,103 adding 'lm_eval/tasks/include/default/Estonian/_estonian_template_yaml' 2026-05-11T15:53:36,104 adding 'lm_eval/tasks/include/default/Estonian/_include_base_44_estonian.yaml' 2026-05-11T15:53:36,105 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_applied_science.yaml' 2026-05-11T15:53:36,106 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_arts_humanities.yaml' 2026-05-11T15:53:36,107 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_health_oriented_education.yaml' 2026-05-11T15:53:36,109 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_social_science.yaml' 2026-05-11T15:53:36,110 adding 'lm_eval/tasks/include/default/Estonian/include_base_44_estonian_stem.yaml' 2026-05-11T15:53:36,111 adding 'lm_eval/tasks/include/default/Estonian/utils.py' 2026-05-11T15:53:36,113 adding 'lm_eval/tasks/include/default/Finnish/_finnish_template_yaml' 2026-05-11T15:53:36,114 adding 'lm_eval/tasks/include/default/Finnish/_include_base_44_finnish.yaml' 2026-05-11T15:53:36,115 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_applied_science.yaml' 2026-05-11T15:53:36,116 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_arts_humanities.yaml' 2026-05-11T15:53:36,117 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_health_oriented_education.yaml' 2026-05-11T15:53:36,119 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_social_science.yaml' 2026-05-11T15:53:36,120 adding 'lm_eval/tasks/include/default/Finnish/include_base_44_finnish_stem.yaml' 2026-05-11T15:53:36,121 adding 'lm_eval/tasks/include/default/Finnish/utils.py' 2026-05-11T15:53:36,123 adding 'lm_eval/tasks/include/default/French/_french_template_yaml' 2026-05-11T15:53:36,124 adding 'lm_eval/tasks/include/default/French/_include_base_44_french.yaml' 2026-05-11T15:53:36,125 adding 'lm_eval/tasks/include/default/French/include_base_44_french_arts_humanities.yaml' 2026-05-11T15:53:36,126 adding 'lm_eval/tasks/include/default/French/include_base_44_french_driving_license.yaml' 2026-05-11T15:53:36,128 adding 'lm_eval/tasks/include/default/French/include_base_44_french_health_oriented_education.yaml' 2026-05-11T15:53:36,129 adding 'lm_eval/tasks/include/default/French/include_base_44_french_social_science.yaml' 2026-05-11T15:53:36,130 adding 'lm_eval/tasks/include/default/French/include_base_44_french_stem.yaml' 2026-05-11T15:53:36,131 adding 'lm_eval/tasks/include/default/French/utils.py' 2026-05-11T15:53:36,133 adding 'lm_eval/tasks/include/default/Georgian/_georgian_template_yaml' 2026-05-11T15:53:36,134 adding 'lm_eval/tasks/include/default/Georgian/_include_base_44_georgian.yaml' 2026-05-11T15:53:36,136 adding 'lm_eval/tasks/include/default/Georgian/include_base_44_georgian_arts_humanities.yaml' 2026-05-11T15:53:36,137 adding 'lm_eval/tasks/include/default/Georgian/utils.py' 2026-05-11T15:53:36,139 adding 'lm_eval/tasks/include/default/German/_german_template_yaml' 2026-05-11T15:53:36,140 adding 'lm_eval/tasks/include/default/German/_include_base_44_german.yaml' 2026-05-11T15:53:36,141 adding 'lm_eval/tasks/include/default/German/include_base_44_german_driving_license.yaml' 2026-05-11T15:53:36,143 adding 'lm_eval/tasks/include/default/German/include_base_44_german_social_science.yaml' 2026-05-11T15:53:36,144 adding 'lm_eval/tasks/include/default/German/include_base_44_german_stem.yaml' 2026-05-11T15:53:36,145 adding 'lm_eval/tasks/include/default/German/utils.py' 2026-05-11T15:53:36,147 adding 'lm_eval/tasks/include/default/Greek/_greek_template_yaml' 2026-05-11T15:53:36,148 adding 'lm_eval/tasks/include/default/Greek/_include_base_44_greek.yaml' 2026-05-11T15:53:36,150 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_arts_humanities.yaml' 2026-05-11T15:53:36,151 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_business_commerce.yaml' 2026-05-11T15:53:36,152 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_health_oriented_education.yaml' 2026-05-11T15:53:36,153 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_medical_license.yaml' 2026-05-11T15:53:36,154 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_professional_certification.yaml' 2026-05-11T15:53:36,155 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_social_science.yaml' 2026-05-11T15:53:36,156 adding 'lm_eval/tasks/include/default/Greek/include_base_44_greek_stem.yaml' 2026-05-11T15:53:36,157 adding 'lm_eval/tasks/include/default/Greek/utils.py' 2026-05-11T15:53:36,159 adding 'lm_eval/tasks/include/default/Hebrew/_hebrew_template_yaml' 2026-05-11T15:53:36,160 adding 'lm_eval/tasks/include/default/Hebrew/_include_base_44_hebrew.yaml' 2026-05-11T15:53:36,162 adding 'lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_arts_humanities.yaml' 2026-05-11T15:53:36,163 adding 'lm_eval/tasks/include/default/Hebrew/include_base_44_hebrew_driving_license.yaml' 2026-05-11T15:53:36,164 adding 'lm_eval/tasks/include/default/Hebrew/utils.py' 2026-05-11T15:53:36,166 adding 'lm_eval/tasks/include/default/Hindi/_hindi_template_yaml' 2026-05-11T15:53:36,167 adding 'lm_eval/tasks/include/default/Hindi/_include_base_44_hindi.yaml' 2026-05-11T15:53:36,168 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_applied_science.yaml' 2026-05-11T15:53:36,169 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_arts_humanities.yaml' 2026-05-11T15:53:36,171 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_driving_license.yaml' 2026-05-11T15:53:36,172 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_general_knowledge.yaml' 2026-05-11T15:53:36,173 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_health_oriented_education.yaml' 2026-05-11T15:53:36,174 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_professional_certification.yaml' 2026-05-11T15:53:36,176 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_social_science.yaml' 2026-05-11T15:53:36,177 adding 'lm_eval/tasks/include/default/Hindi/include_base_44_hindi_stem.yaml' 2026-05-11T15:53:36,178 adding 'lm_eval/tasks/include/default/Hindi/utils.py' 2026-05-11T15:53:36,180 adding 'lm_eval/tasks/include/default/Hungarian/_hungarian_template_yaml' 2026-05-11T15:53:36,181 adding 'lm_eval/tasks/include/default/Hungarian/_include_base_44_hungarian.yaml' 2026-05-11T15:53:36,183 adding 'lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_applied_science.yaml' 2026-05-11T15:53:36,184 adding 'lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_social_science.yaml' 2026-05-11T15:53:36,185 adding 'lm_eval/tasks/include/default/Hungarian/include_base_44_hungarian_stem.yaml' 2026-05-11T15:53:36,186 adding 'lm_eval/tasks/include/default/Hungarian/utils.py' 2026-05-11T15:53:36,188 adding 'lm_eval/tasks/include/default/Indonesian/_include_base_44_indonesian.yaml' 2026-05-11T15:53:36,190 adding 'lm_eval/tasks/include/default/Indonesian/_indonesian_template_yaml' 2026-05-11T15:53:36,191 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_applied_science.yaml' 2026-05-11T15:53:36,192 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_arts_humanities.yaml' 2026-05-11T15:53:36,193 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_professional_certification.yaml' 2026-05-11T15:53:36,194 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_social_science.yaml' 2026-05-11T15:53:36,196 adding 'lm_eval/tasks/include/default/Indonesian/include_base_44_indonesian_stem.yaml' 2026-05-11T15:53:36,197 adding 'lm_eval/tasks/include/default/Indonesian/utils.py' 2026-05-11T15:53:36,199 adding 'lm_eval/tasks/include/default/Italian/_include_base_44_italian.yaml' 2026-05-11T15:53:36,200 adding 'lm_eval/tasks/include/default/Italian/_italian_template_yaml' 2026-05-11T15:53:36,201 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_applied_science.yaml' 2026-05-11T15:53:36,202 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_arts_humanities.yaml' 2026-05-11T15:53:36,203 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_health_oriented_education.yaml' 2026-05-11T15:53:36,204 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_professional_certification.yaml' 2026-05-11T15:53:36,205 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_social_science.yaml' 2026-05-11T15:53:36,206 adding 'lm_eval/tasks/include/default/Italian/include_base_44_italian_stem.yaml' 2026-05-11T15:53:36,208 adding 'lm_eval/tasks/include/default/Italian/utils.py' 2026-05-11T15:53:36,209 adding 'lm_eval/tasks/include/default/Japanese/_include_base_44_japanese.yaml' 2026-05-11T15:53:36,211 adding 'lm_eval/tasks/include/default/Japanese/_japanese_template_yaml' 2026-05-11T15:53:36,212 adding 'lm_eval/tasks/include/default/Japanese/include_base_44_japanese_driving_license.yaml' 2026-05-11T15:53:36,213 adding 'lm_eval/tasks/include/default/Japanese/include_base_44_japanese_medical_license.yaml' 2026-05-11T15:53:36,214 adding 'lm_eval/tasks/include/default/Japanese/include_base_44_japanese_professional_certification.yaml' 2026-05-11T15:53:36,215 adding 'lm_eval/tasks/include/default/Japanese/utils.py' 2026-05-11T15:53:36,217 adding 'lm_eval/tasks/include/default/Kazakh/_include_base_44_kazakh.yaml' 2026-05-11T15:53:36,218 adding 'lm_eval/tasks/include/default/Kazakh/_kazakh_template_yaml' 2026-05-11T15:53:36,219 adding 'lm_eval/tasks/include/default/Kazakh/include_base_44_kazakh_arts_humanities.yaml' 2026-05-11T15:53:36,220 adding 'lm_eval/tasks/include/default/Kazakh/utils.py' 2026-05-11T15:53:36,222 adding 'lm_eval/tasks/include/default/Korean/_include_base_44_korean.yaml' 2026-05-11T15:53:36,223 adding 'lm_eval/tasks/include/default/Korean/_korean_template_yaml' 2026-05-11T15:53:36,225 adding 'lm_eval/tasks/include/default/Korean/include_base_44_korean_professional_certification.yaml' 2026-05-11T15:53:36,226 adding 'lm_eval/tasks/include/default/Korean/include_base_44_korean_social_science.yaml' 2026-05-11T15:53:36,227 adding 'lm_eval/tasks/include/default/Korean/utils.py' 2026-05-11T15:53:36,229 adding 'lm_eval/tasks/include/default/Lithuanian/_include_base_44_lithuanian.yaml' 2026-05-11T15:53:36,230 adding 'lm_eval/tasks/include/default/Lithuanian/_lithuanian_template_yaml' 2026-05-11T15:53:36,231 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml' 2026-05-11T15:53:36,233 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_business_commerce.yaml' 2026-05-11T15:53:36,234 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_professional_certification.yaml' 2026-05-11T15:53:36,235 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_social_science.yaml' 2026-05-11T15:53:36,236 adding 'lm_eval/tasks/include/default/Lithuanian/include_base_44_lithuanian_stem.yaml' 2026-05-11T15:53:36,237 adding 'lm_eval/tasks/include/default/Lithuanian/utils.py' 2026-05-11T15:53:36,239 adding 'lm_eval/tasks/include/default/Malay/_include_base_44_malay.yaml' 2026-05-11T15:53:36,241 adding 'lm_eval/tasks/include/default/Malay/_malay_template_yaml' 2026-05-11T15:53:36,242 adding 'lm_eval/tasks/include/default/Malay/include_base_44_malay_arts_humanities.yaml' 2026-05-11T15:53:36,243 adding 'lm_eval/tasks/include/default/Malay/include_base_44_malay_business_commerce.yaml' 2026-05-11T15:53:36,244 adding 'lm_eval/tasks/include/default/Malay/include_base_44_malay_social_science.yaml' 2026-05-11T15:53:36,245 adding 'lm_eval/tasks/include/default/Malay/utils.py' 2026-05-11T15:53:36,247 adding 'lm_eval/tasks/include/default/Malayalam/_include_base_44_malayalam.yaml' 2026-05-11T15:53:36,248 adding 'lm_eval/tasks/include/default/Malayalam/_malayalam_template_yaml' 2026-05-11T15:53:36,249 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_arts_humanities.yaml' 2026-05-11T15:53:36,250 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_general_knowledge.yaml' 2026-05-11T15:53:36,251 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_health_oriented_education.yaml' 2026-05-11T15:53:36,252 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_marine_license.yaml' 2026-05-11T15:53:36,254 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_social_science.yaml' 2026-05-11T15:53:36,255 adding 'lm_eval/tasks/include/default/Malayalam/include_base_44_malayalam_stem.yaml' 2026-05-11T15:53:36,256 adding 'lm_eval/tasks/include/default/Malayalam/utils.py' 2026-05-11T15:53:36,258 adding 'lm_eval/tasks/include/default/Nepali/_include_base_44_nepali.yaml' 2026-05-11T15:53:36,259 adding 'lm_eval/tasks/include/default/Nepali/_nepali_template_yaml' 2026-05-11T15:53:36,260 adding 'lm_eval/tasks/include/default/Nepali/include_base_44_nepali_driving_license.yaml' 2026-05-11T15:53:36,261 adding 'lm_eval/tasks/include/default/Nepali/include_base_44_nepali_professional_certification.yaml' 2026-05-11T15:53:36,262 adding 'lm_eval/tasks/include/default/Nepali/utils.py' 2026-05-11T15:53:36,264 adding 'lm_eval/tasks/include/default/North Macedonian/_include_base_44_north macedonian.yaml' 2026-05-11T15:53:36,265 adding 'lm_eval/tasks/include/default/North Macedonian/_north macedonian_template_yaml' 2026-05-11T15:53:36,266 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml' 2026-05-11T15:53:36,268 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_business_commerce.yaml' 2026-05-11T15:53:36,269 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_social_science.yaml' 2026-05-11T15:53:36,270 adding 'lm_eval/tasks/include/default/North Macedonian/include_base_44_north macedonian_stem.yaml' 2026-05-11T15:53:36,271 adding 'lm_eval/tasks/include/default/North Macedonian/utils.py' 2026-05-11T15:53:36,273 adding 'lm_eval/tasks/include/default/Persian/_include_base_44_persian.yaml' 2026-05-11T15:53:36,274 adding 'lm_eval/tasks/include/default/Persian/_persian_template_yaml' 2026-05-11T15:53:36,276 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_arts_humanities.yaml' 2026-05-11T15:53:36,277 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_driving_license.yaml' 2026-05-11T15:53:36,278 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_professional_certification.yaml' 2026-05-11T15:53:36,279 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_social_science.yaml' 2026-05-11T15:53:36,280 adding 'lm_eval/tasks/include/default/Persian/include_base_44_persian_stem.yaml' 2026-05-11T15:53:36,281 adding 'lm_eval/tasks/include/default/Persian/utils.py' 2026-05-11T15:53:36,283 adding 'lm_eval/tasks/include/default/Polish/_include_base_44_polish.yaml' 2026-05-11T15:53:36,284 adding 'lm_eval/tasks/include/default/Polish/_polish_template_yaml' 2026-05-11T15:53:36,286 adding 'lm_eval/tasks/include/default/Polish/include_base_44_polish_professional_certification.yaml' 2026-05-11T15:53:36,287 adding 'lm_eval/tasks/include/default/Polish/include_base_44_polish_social_science.yaml' 2026-05-11T15:53:36,288 adding 'lm_eval/tasks/include/default/Polish/include_base_44_polish_stem.yaml' 2026-05-11T15:53:36,289 adding 'lm_eval/tasks/include/default/Polish/utils.py' 2026-05-11T15:53:36,291 adding 'lm_eval/tasks/include/default/Portuguese/_include_base_44_portuguese.yaml' 2026-05-11T15:53:36,293 adding 'lm_eval/tasks/include/default/Portuguese/_portuguese_template_yaml' 2026-05-11T15:53:36,294 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_applied_science.yaml' 2026-05-11T15:53:36,295 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_arts_humanities.yaml' 2026-05-11T15:53:36,296 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_business_commerce.yaml' 2026-05-11T15:53:36,297 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_health_oriented_education.yaml' 2026-05-11T15:53:36,298 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_social_science.yaml' 2026-05-11T15:53:36,299 adding 'lm_eval/tasks/include/default/Portuguese/include_base_44_portuguese_stem.yaml' 2026-05-11T15:53:36,300 adding 'lm_eval/tasks/include/default/Portuguese/utils.py' 2026-05-11T15:53:36,302 adding 'lm_eval/tasks/include/default/Russian/_include_base_44_russian.yaml' 2026-05-11T15:53:36,303 adding 'lm_eval/tasks/include/default/Russian/_russian_template_yaml' 2026-05-11T15:53:36,304 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_applied_science.yaml' 2026-05-11T15:53:36,306 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_arts_humanities.yaml' 2026-05-11T15:53:36,307 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_business_commerce.yaml' 2026-05-11T15:53:36,308 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_driving_license.yaml' 2026-05-11T15:53:36,309 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_health_oriented_education.yaml' 2026-05-11T15:53:36,310 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_marine_license.yaml' 2026-05-11T15:53:36,311 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_social_science.yaml' 2026-05-11T15:53:36,312 adding 'lm_eval/tasks/include/default/Russian/include_base_44_russian_stem.yaml' 2026-05-11T15:53:36,314 adding 'lm_eval/tasks/include/default/Russian/utils.py' 2026-05-11T15:53:36,316 adding 'lm_eval/tasks/include/default/Serbian/_include_base_44_serbian.yaml' 2026-05-11T15:53:36,317 adding 'lm_eval/tasks/include/default/Serbian/_serbian_template_yaml' 2026-05-11T15:53:36,318 adding 'lm_eval/tasks/include/default/Serbian/include_base_44_serbian_arts_humanities.yaml' 2026-05-11T15:53:36,319 adding 'lm_eval/tasks/include/default/Serbian/include_base_44_serbian_social_science.yaml' 2026-05-11T15:53:36,320 adding 'lm_eval/tasks/include/default/Serbian/include_base_44_serbian_stem.yaml' 2026-05-11T15:53:36,321 adding 'lm_eval/tasks/include/default/Serbian/utils.py' 2026-05-11T15:53:36,323 adding 'lm_eval/tasks/include/default/Spanish/_include_base_44_spanish.yaml' 2026-05-11T15:53:36,324 adding 'lm_eval/tasks/include/default/Spanish/_spanish_template_yaml' 2026-05-11T15:53:36,325 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_arts_humanities.yaml' 2026-05-11T15:53:36,327 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_health_oriented_education.yaml' 2026-05-11T15:53:36,328 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_social_science.yaml' 2026-05-11T15:53:36,329 adding 'lm_eval/tasks/include/default/Spanish/include_base_44_spanish_stem.yaml' 2026-05-11T15:53:36,330 adding 'lm_eval/tasks/include/default/Spanish/utils.py' 2026-05-11T15:53:36,332 adding 'lm_eval/tasks/include/default/Tagalog/_include_base_44_tagalog.yaml' 2026-05-11T15:53:36,333 adding 'lm_eval/tasks/include/default/Tagalog/_tagalog_template_yaml' 2026-05-11T15:53:36,335 adding 'lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_arts_humanities.yaml' 2026-05-11T15:53:36,336 adding 'lm_eval/tasks/include/default/Tagalog/include_base_44_tagalog_driving_license.yaml' 2026-05-11T15:53:36,337 adding 'lm_eval/tasks/include/default/Tagalog/utils.py' 2026-05-11T15:53:36,339 adding 'lm_eval/tasks/include/default/Tamil/_include_base_44_tamil.yaml' 2026-05-11T15:53:36,340 adding 'lm_eval/tasks/include/default/Tamil/_tamil_template_yaml' 2026-05-11T15:53:36,341 adding 'lm_eval/tasks/include/default/Tamil/include_base_44_tamil_general_knowledge.yaml' 2026-05-11T15:53:36,342 adding 'lm_eval/tasks/include/default/Tamil/include_base_44_tamil_stem.yaml' 2026-05-11T15:53:36,343 adding 'lm_eval/tasks/include/default/Tamil/utils.py' 2026-05-11T15:53:36,345 adding 'lm_eval/tasks/include/default/Telugu/_include_base_44_telugu.yaml' 2026-05-11T15:53:36,346 adding 'lm_eval/tasks/include/default/Telugu/_telugu_template_yaml' 2026-05-11T15:53:36,347 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_applied_science.yaml' 2026-05-11T15:53:36,348 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_arts_humanities.yaml' 2026-05-11T15:53:36,349 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_social_science.yaml' 2026-05-11T15:53:36,351 adding 'lm_eval/tasks/include/default/Telugu/include_base_44_telugu_stem.yaml' 2026-05-11T15:53:36,352 adding 'lm_eval/tasks/include/default/Telugu/utils.py' 2026-05-11T15:53:36,354 adding 'lm_eval/tasks/include/default/Turkish/_include_base_44_turkish.yaml' 2026-05-11T15:53:36,355 adding 'lm_eval/tasks/include/default/Turkish/_turkish_template_yaml' 2026-05-11T15:53:36,356 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_arts_humanities.yaml' 2026-05-11T15:53:36,357 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_business_commerce.yaml' 2026-05-11T15:53:36,358 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_social_science.yaml' 2026-05-11T15:53:36,360 adding 'lm_eval/tasks/include/default/Turkish/include_base_44_turkish_stem.yaml' 2026-05-11T15:53:36,361 adding 'lm_eval/tasks/include/default/Turkish/utils.py' 2026-05-11T15:53:36,363 adding 'lm_eval/tasks/include/default/Ukrainian/_include_base_44_ukrainian.yaml' 2026-05-11T15:53:36,364 adding 'lm_eval/tasks/include/default/Ukrainian/_ukrainian_template_yaml' 2026-05-11T15:53:36,365 adding 'lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml' 2026-05-11T15:53:36,367 adding 'lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_social_science.yaml' 2026-05-11T15:53:36,368 adding 'lm_eval/tasks/include/default/Ukrainian/include_base_44_ukrainian_stem.yaml' 2026-05-11T15:53:36,369 adding 'lm_eval/tasks/include/default/Ukrainian/utils.py' 2026-05-11T15:53:36,371 adding 'lm_eval/tasks/include/default/Urdu/_include_base_44_urdu.yaml' 2026-05-11T15:53:36,372 adding 'lm_eval/tasks/include/default/Urdu/_urdu_template_yaml' 2026-05-11T15:53:36,373 adding 'lm_eval/tasks/include/default/Urdu/include_base_44_urdu_arts_humanities.yaml' 2026-05-11T15:53:36,375 adding 'lm_eval/tasks/include/default/Urdu/include_base_44_urdu_health_oriented_education.yaml' 2026-05-11T15:53:36,376 adding 'lm_eval/tasks/include/default/Urdu/include_base_44_urdu_stem.yaml' 2026-05-11T15:53:36,377 adding 'lm_eval/tasks/include/default/Urdu/utils.py' 2026-05-11T15:53:36,379 adding 'lm_eval/tasks/include/default/Uzbek/_include_base_44_uzbek.yaml' 2026-05-11T15:53:36,380 adding 'lm_eval/tasks/include/default/Uzbek/_uzbek_template_yaml' 2026-05-11T15:53:36,381 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_arts_humanities.yaml' 2026-05-11T15:53:36,383 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_medical_license.yaml' 2026-05-11T15:53:36,384 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_social_science.yaml' 2026-05-11T15:53:36,385 adding 'lm_eval/tasks/include/default/Uzbek/include_base_44_uzbek_stem.yaml' 2026-05-11T15:53:36,386 adding 'lm_eval/tasks/include/default/Uzbek/utils.py' 2026-05-11T15:53:36,388 adding 'lm_eval/tasks/include/default/Vietnamese/_include_base_44_vietnamese.yaml' 2026-05-11T15:53:36,389 adding 'lm_eval/tasks/include/default/Vietnamese/_vietnamese_template_yaml' 2026-05-11T15:53:36,391 adding 'lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml' 2026-05-11T15:53:36,392 adding 'lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_social_science.yaml' 2026-05-11T15:53:36,393 adding 'lm_eval/tasks/include/default/Vietnamese/include_base_44_vietnamese_stem.yaml' 2026-05-11T15:53:36,394 adding 'lm_eval/tasks/include/default/Vietnamese/utils.py' 2026-05-11T15:53:36,397 adding 'lm_eval/tasks/include/few_shot_en/Albanian/_albanian_few_shot_en_template_yaml' 2026-05-11T15:53:36,398 adding 'lm_eval/tasks/include/few_shot_en/Albanian/_include_base_44_albanian.yaml' 2026-05-11T15:53:36,399 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_arts_humanities.yaml' 2026-05-11T15:53:36,400 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_business_commerce.yaml' 2026-05-11T15:53:36,402 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_health_oriented_education.yaml' 2026-05-11T15:53:36,403 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_social_science.yaml' 2026-05-11T15:53:36,404 adding 'lm_eval/tasks/include/few_shot_en/Albanian/include_base_44_albanian_stem.yaml' 2026-05-11T15:53:36,405 adding 'lm_eval/tasks/include/few_shot_en/Albanian/utils.py' 2026-05-11T15:53:36,407 adding 'lm_eval/tasks/include/few_shot_en/Arabic/_arabic_few_shot_en_template_yaml' 2026-05-11T15:53:36,408 adding 'lm_eval/tasks/include/few_shot_en/Arabic/_include_base_44_arabic.yaml' 2026-05-11T15:53:36,410 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_arts_humanities.yaml' 2026-05-11T15:53:36,411 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_business_commerce.yaml' 2026-05-11T15:53:36,412 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_driving_license.yaml' 2026-05-11T15:53:36,413 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_general_knowledge.yaml' 2026-05-11T15:53:36,414 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_social_science.yaml' 2026-05-11T15:53:36,415 adding 'lm_eval/tasks/include/few_shot_en/Arabic/include_base_44_arabic_stem.yaml' 2026-05-11T15:53:36,417 adding 'lm_eval/tasks/include/few_shot_en/Arabic/utils.py' 2026-05-11T15:53:36,418 adding 'lm_eval/tasks/include/few_shot_en/Armenian/_armenian_few_shot_en_template_yaml' 2026-05-11T15:53:36,420 adding 'lm_eval/tasks/include/few_shot_en/Armenian/_include_base_44_armenian.yaml' 2026-05-11T15:53:36,421 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_arts_humanities.yaml' 2026-05-11T15:53:36,422 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_driving_license.yaml' 2026-05-11T15:53:36,423 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_social_science.yaml' 2026-05-11T15:53:36,424 adding 'lm_eval/tasks/include/few_shot_en/Armenian/include_base_44_armenian_stem.yaml' 2026-05-11T15:53:36,426 adding 'lm_eval/tasks/include/few_shot_en/Armenian/utils.py' 2026-05-11T15:53:36,427 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/_azerbaijani_few_shot_en_template_yaml' 2026-05-11T15:53:36,429 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/_include_base_44_azerbaijani.yaml' 2026-05-11T15:53:36,430 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml' 2026-05-11T15:53:36,431 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml' 2026-05-11T15:53:36,432 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml' 2026-05-11T15:53:36,433 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml' 2026-05-11T15:53:36,435 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_social_science.yaml' 2026-05-11T15:53:36,436 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/include_base_44_azerbaijani_stem.yaml' 2026-05-11T15:53:36,437 adding 'lm_eval/tasks/include/few_shot_en/Azerbaijani/utils.py' 2026-05-11T15:53:36,439 adding 'lm_eval/tasks/include/few_shot_en/Basque/_basque_few_shot_en_template_yaml' 2026-05-11T15:53:36,440 adding 'lm_eval/tasks/include/few_shot_en/Basque/_include_base_44_basque.yaml' 2026-05-11T15:53:36,441 adding 'lm_eval/tasks/include/few_shot_en/Basque/include_base_44_basque_professional_certification.yaml' 2026-05-11T15:53:36,443 adding 'lm_eval/tasks/include/few_shot_en/Basque/utils.py' 2026-05-11T15:53:36,444 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/_belarusian_few_shot_en_template_yaml' 2026-05-11T15:53:36,446 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/_include_base_44_belarusian.yaml' 2026-05-11T15:53:36,447 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_arts_humanities.yaml' 2026-05-11T15:53:36,448 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_social_science.yaml' 2026-05-11T15:53:36,449 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/include_base_44_belarusian_stem.yaml' 2026-05-11T15:53:36,451 adding 'lm_eval/tasks/include/few_shot_en/Belarusian/utils.py' 2026-05-11T15:53:36,452 adding 'lm_eval/tasks/include/few_shot_en/Bengali/_bengali_few_shot_en_template_yaml' 2026-05-11T15:53:36,454 adding 'lm_eval/tasks/include/few_shot_en/Bengali/_include_base_44_bengali.yaml' 2026-05-11T15:53:36,455 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_arts_humanities.yaml' 2026-05-11T15:53:36,456 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_general_knowledge.yaml' 2026-05-11T15:53:36,457 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_professional_certification.yaml' 2026-05-11T15:53:36,458 adding 'lm_eval/tasks/include/few_shot_en/Bengali/include_base_44_bengali_stem.yaml' 2026-05-11T15:53:36,459 adding 'lm_eval/tasks/include/few_shot_en/Bengali/utils.py' 2026-05-11T15:53:36,461 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/_bulgarian_few_shot_en_template_yaml' 2026-05-11T15:53:36,462 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/_include_base_44_bulgarian.yaml' 2026-05-11T15:53:36,464 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml' 2026-05-11T15:53:36,465 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_social_science.yaml' 2026-05-11T15:53:36,466 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/include_base_44_bulgarian_stem.yaml' 2026-05-11T15:53:36,467 adding 'lm_eval/tasks/include/few_shot_en/Bulgarian/utils.py' 2026-05-11T15:53:36,469 adding 'lm_eval/tasks/include/few_shot_en/Chinese/_chinese_few_shot_en_template_yaml' 2026-05-11T15:53:36,470 adding 'lm_eval/tasks/include/few_shot_en/Chinese/_include_base_44_chinese.yaml' 2026-05-11T15:53:36,472 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_applied_science.yaml' 2026-05-11T15:53:36,473 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_arts_humanities.yaml' 2026-05-11T15:53:36,474 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_business_commerce.yaml' 2026-05-11T15:53:36,475 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_driving_license.yaml' 2026-05-11T15:53:36,476 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_health_oriented_education.yaml' 2026-05-11T15:53:36,478 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_professional_certification.yaml' 2026-05-11T15:53:36,479 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_social_science.yaml' 2026-05-11T15:53:36,480 adding 'lm_eval/tasks/include/few_shot_en/Chinese/include_base_44_chinese_stem.yaml' 2026-05-11T15:53:36,481 adding 'lm_eval/tasks/include/few_shot_en/Chinese/utils.py' 2026-05-11T15:53:36,483 adding 'lm_eval/tasks/include/few_shot_en/Croatian/_croatian_few_shot_en_template_yaml' 2026-05-11T15:53:36,484 adding 'lm_eval/tasks/include/few_shot_en/Croatian/_include_base_44_croatian.yaml' 2026-05-11T15:53:36,485 adding 'lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_arts_humanities.yaml' 2026-05-11T15:53:36,486 adding 'lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_social_science.yaml' 2026-05-11T15:53:36,487 adding 'lm_eval/tasks/include/few_shot_en/Croatian/include_base_44_croatian_stem.yaml' 2026-05-11T15:53:36,489 adding 'lm_eval/tasks/include/few_shot_en/Croatian/utils.py' 2026-05-11T15:53:36,490 adding 'lm_eval/tasks/include/few_shot_en/Dutch/_dutch_few_shot_en_template_yaml' 2026-05-11T15:53:36,491 adding 'lm_eval/tasks/include/few_shot_en/Dutch/_include_base_44_dutch.yaml' 2026-05-11T15:53:36,492 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_applied_science.yaml' 2026-05-11T15:53:36,494 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_arts_humanities.yaml' 2026-05-11T15:53:36,495 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_health_oriented_education.yaml' 2026-05-11T15:53:36,496 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_social_science.yaml' 2026-05-11T15:53:36,497 adding 'lm_eval/tasks/include/few_shot_en/Dutch/include_base_44_dutch_stem.yaml' 2026-05-11T15:53:36,498 adding 'lm_eval/tasks/include/few_shot_en/Dutch/utils.py' 2026-05-11T15:53:36,500 adding 'lm_eval/tasks/include/few_shot_en/Estonian/_estonian_few_shot_en_template_yaml' 2026-05-11T15:53:36,501 adding 'lm_eval/tasks/include/few_shot_en/Estonian/_include_base_44_estonian.yaml' 2026-05-11T15:53:36,502 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_applied_science.yaml' 2026-05-11T15:53:36,503 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_arts_humanities.yaml' 2026-05-11T15:53:36,504 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_health_oriented_education.yaml' 2026-05-11T15:53:36,506 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_social_science.yaml' 2026-05-11T15:53:36,507 adding 'lm_eval/tasks/include/few_shot_en/Estonian/include_base_44_estonian_stem.yaml' 2026-05-11T15:53:36,508 adding 'lm_eval/tasks/include/few_shot_en/Estonian/utils.py' 2026-05-11T15:53:36,510 adding 'lm_eval/tasks/include/few_shot_en/Finnish/_finnish_few_shot_en_template_yaml' 2026-05-11T15:53:36,511 adding 'lm_eval/tasks/include/few_shot_en/Finnish/_include_base_44_finnish.yaml' 2026-05-11T15:53:36,512 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_applied_science.yaml' 2026-05-11T15:53:36,514 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_arts_humanities.yaml' 2026-05-11T15:53:36,515 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_health_oriented_education.yaml' 2026-05-11T15:53:36,516 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_social_science.yaml' 2026-05-11T15:53:36,517 adding 'lm_eval/tasks/include/few_shot_en/Finnish/include_base_44_finnish_stem.yaml' 2026-05-11T15:53:36,519 adding 'lm_eval/tasks/include/few_shot_en/Finnish/utils.py' 2026-05-11T15:53:36,521 adding 'lm_eval/tasks/include/few_shot_en/French/_french_few_shot_en_template_yaml' 2026-05-11T15:53:36,522 adding 'lm_eval/tasks/include/few_shot_en/French/_include_base_44_french.yaml' 2026-05-11T15:53:36,523 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_arts_humanities.yaml' 2026-05-11T15:53:36,524 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_driving_license.yaml' 2026-05-11T15:53:36,525 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_health_oriented_education.yaml' 2026-05-11T15:53:36,527 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_social_science.yaml' 2026-05-11T15:53:36,528 adding 'lm_eval/tasks/include/few_shot_en/French/include_base_44_french_stem.yaml' 2026-05-11T15:53:36,529 adding 'lm_eval/tasks/include/few_shot_en/French/utils.py' 2026-05-11T15:53:36,531 adding 'lm_eval/tasks/include/few_shot_en/Georgian/_georgian_few_shot_en_template_yaml' 2026-05-11T15:53:36,532 adding 'lm_eval/tasks/include/few_shot_en/Georgian/_include_base_44_georgian.yaml' 2026-05-11T15:53:36,533 adding 'lm_eval/tasks/include/few_shot_en/Georgian/include_base_44_georgian_arts_humanities.yaml' 2026-05-11T15:53:36,534 adding 'lm_eval/tasks/include/few_shot_en/Georgian/utils.py' 2026-05-11T15:53:36,536 adding 'lm_eval/tasks/include/few_shot_en/German/_german_few_shot_en_template_yaml' 2026-05-11T15:53:36,537 adding 'lm_eval/tasks/include/few_shot_en/German/_include_base_44_german.yaml' 2026-05-11T15:53:36,538 adding 'lm_eval/tasks/include/few_shot_en/German/include_base_44_german_driving_license.yaml' 2026-05-11T15:53:36,539 adding 'lm_eval/tasks/include/few_shot_en/German/include_base_44_german_social_science.yaml' 2026-05-11T15:53:36,541 adding 'lm_eval/tasks/include/few_shot_en/German/include_base_44_german_stem.yaml' 2026-05-11T15:53:36,542 adding 'lm_eval/tasks/include/few_shot_en/German/utils.py' 2026-05-11T15:53:36,544 adding 'lm_eval/tasks/include/few_shot_en/Greek/_greek_few_shot_en_template_yaml' 2026-05-11T15:53:36,545 adding 'lm_eval/tasks/include/few_shot_en/Greek/_include_base_44_greek.yaml' 2026-05-11T15:53:36,546 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_arts_humanities.yaml' 2026-05-11T15:53:36,547 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_business_commerce.yaml' 2026-05-11T15:53:36,548 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_health_oriented_education.yaml' 2026-05-11T15:53:36,549 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_medical_license.yaml' 2026-05-11T15:53:36,550 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_professional_certification.yaml' 2026-05-11T15:53:36,552 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_social_science.yaml' 2026-05-11T15:53:36,553 adding 'lm_eval/tasks/include/few_shot_en/Greek/include_base_44_greek_stem.yaml' 2026-05-11T15:53:36,554 adding 'lm_eval/tasks/include/few_shot_en/Greek/utils.py' 2026-05-11T15:53:36,556 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/_hebrew_few_shot_en_template_yaml' 2026-05-11T15:53:36,557 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/_include_base_44_hebrew.yaml' 2026-05-11T15:53:36,558 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_arts_humanities.yaml' 2026-05-11T15:53:36,559 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/include_base_44_hebrew_driving_license.yaml' 2026-05-11T15:53:36,561 adding 'lm_eval/tasks/include/few_shot_en/Hebrew/utils.py' 2026-05-11T15:53:36,563 adding 'lm_eval/tasks/include/few_shot_en/Hindi/_hindi_few_shot_en_template_yaml' 2026-05-11T15:53:36,564 adding 'lm_eval/tasks/include/few_shot_en/Hindi/_include_base_44_hindi.yaml' 2026-05-11T15:53:36,565 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_applied_science.yaml' 2026-05-11T15:53:36,566 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_arts_humanities.yaml' 2026-05-11T15:53:36,567 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_driving_license.yaml' 2026-05-11T15:53:36,569 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_general_knowledge.yaml' 2026-05-11T15:53:36,570 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_health_oriented_education.yaml' 2026-05-11T15:53:36,571 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_professional_certification.yaml' 2026-05-11T15:53:36,573 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_social_science.yaml' 2026-05-11T15:53:36,574 adding 'lm_eval/tasks/include/few_shot_en/Hindi/include_base_44_hindi_stem.yaml' 2026-05-11T15:53:36,575 adding 'lm_eval/tasks/include/few_shot_en/Hindi/utils.py' 2026-05-11T15:53:36,577 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/_hungarian_few_shot_en_template_yaml' 2026-05-11T15:53:36,578 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/_include_base_44_hungarian.yaml' 2026-05-11T15:53:36,579 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_applied_science.yaml' 2026-05-11T15:53:36,580 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_social_science.yaml' 2026-05-11T15:53:36,581 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/include_base_44_hungarian_stem.yaml' 2026-05-11T15:53:36,582 adding 'lm_eval/tasks/include/few_shot_en/Hungarian/utils.py' 2026-05-11T15:53:36,584 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/_include_base_44_indonesian.yaml' 2026-05-11T15:53:36,585 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/_indonesian_few_shot_en_template_yaml' 2026-05-11T15:53:36,586 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_applied_science.yaml' 2026-05-11T15:53:36,588 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_arts_humanities.yaml' 2026-05-11T15:53:36,589 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_professional_certification.yaml' 2026-05-11T15:53:36,590 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_social_science.yaml' 2026-05-11T15:53:36,591 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/include_base_44_indonesian_stem.yaml' 2026-05-11T15:53:36,592 adding 'lm_eval/tasks/include/few_shot_en/Indonesian/utils.py' 2026-05-11T15:53:36,594 adding 'lm_eval/tasks/include/few_shot_en/Italian/_include_base_44_italian.yaml' 2026-05-11T15:53:36,596 adding 'lm_eval/tasks/include/few_shot_en/Italian/_italian_few_shot_en_template_yaml' 2026-05-11T15:53:36,597 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_applied_science.yaml' 2026-05-11T15:53:36,598 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_arts_humanities.yaml' 2026-05-11T15:53:36,599 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_health_oriented_education.yaml' 2026-05-11T15:53:36,600 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_professional_certification.yaml' 2026-05-11T15:53:36,601 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_social_science.yaml' 2026-05-11T15:53:36,603 adding 'lm_eval/tasks/include/few_shot_en/Italian/include_base_44_italian_stem.yaml' 2026-05-11T15:53:36,604 adding 'lm_eval/tasks/include/few_shot_en/Italian/utils.py' 2026-05-11T15:53:36,606 adding 'lm_eval/tasks/include/few_shot_en/Japanese/_include_base_44_japanese.yaml' 2026-05-11T15:53:36,607 adding 'lm_eval/tasks/include/few_shot_en/Japanese/_japanese_few_shot_en_template_yaml' 2026-05-11T15:53:36,608 adding 'lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_driving_license.yaml' 2026-05-11T15:53:36,609 adding 'lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_medical_license.yaml' 2026-05-11T15:53:36,611 adding 'lm_eval/tasks/include/few_shot_en/Japanese/include_base_44_japanese_professional_certification.yaml' 2026-05-11T15:53:36,612 adding 'lm_eval/tasks/include/few_shot_en/Japanese/utils.py' 2026-05-11T15:53:36,614 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/_include_base_44_kazakh.yaml' 2026-05-11T15:53:36,615 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/_kazakh_few_shot_en_template_yaml' 2026-05-11T15:53:36,616 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/include_base_44_kazakh_arts_humanities.yaml' 2026-05-11T15:53:36,618 adding 'lm_eval/tasks/include/few_shot_en/Kazakh/utils.py' 2026-05-11T15:53:36,619 adding 'lm_eval/tasks/include/few_shot_en/Korean/_include_base_44_korean.yaml' 2026-05-11T15:53:36,621 adding 'lm_eval/tasks/include/few_shot_en/Korean/_korean_few_shot_en_template_yaml' 2026-05-11T15:53:36,622 adding 'lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_professional_certification.yaml' 2026-05-11T15:53:36,623 adding 'lm_eval/tasks/include/few_shot_en/Korean/include_base_44_korean_social_science.yaml' 2026-05-11T15:53:36,624 adding 'lm_eval/tasks/include/few_shot_en/Korean/utils.py' 2026-05-11T15:53:36,626 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/_include_base_44_lithuanian.yaml' 2026-05-11T15:53:36,627 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/_lithuanian_few_shot_en_template_yaml' 2026-05-11T15:53:36,628 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml' 2026-05-11T15:53:36,630 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_business_commerce.yaml' 2026-05-11T15:53:36,631 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_professional_certification.yaml' 2026-05-11T15:53:36,632 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_social_science.yaml' 2026-05-11T15:53:36,633 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/include_base_44_lithuanian_stem.yaml' 2026-05-11T15:53:36,634 adding 'lm_eval/tasks/include/few_shot_en/Lithuanian/utils.py' 2026-05-11T15:53:36,636 adding 'lm_eval/tasks/include/few_shot_en/Malay/_include_base_44_malay.yaml' 2026-05-11T15:53:36,638 adding 'lm_eval/tasks/include/few_shot_en/Malay/_malay_few_shot_en_template_yaml' 2026-05-11T15:53:36,639 adding 'lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_arts_humanities.yaml' 2026-05-11T15:53:36,640 adding 'lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_business_commerce.yaml' 2026-05-11T15:53:36,641 adding 'lm_eval/tasks/include/few_shot_en/Malay/include_base_44_malay_social_science.yaml' 2026-05-11T15:53:36,642 adding 'lm_eval/tasks/include/few_shot_en/Malay/utils.py' 2026-05-11T15:53:36,644 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/_include_base_44_malayalam.yaml' 2026-05-11T15:53:36,645 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/_malayalam_few_shot_en_template_yaml' 2026-05-11T15:53:36,647 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_arts_humanities.yaml' 2026-05-11T15:53:36,648 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_general_knowledge.yaml' 2026-05-11T15:53:36,649 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_health_oriented_education.yaml' 2026-05-11T15:53:36,650 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_marine_license.yaml' 2026-05-11T15:53:36,651 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_social_science.yaml' 2026-05-11T15:53:36,652 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/include_base_44_malayalam_stem.yaml' 2026-05-11T15:53:36,654 adding 'lm_eval/tasks/include/few_shot_en/Malayalam/utils.py' 2026-05-11T15:53:36,656 adding 'lm_eval/tasks/include/few_shot_en/Nepali/_include_base_44_nepali.yaml' 2026-05-11T15:53:36,657 adding 'lm_eval/tasks/include/few_shot_en/Nepali/_nepali_few_shot_en_template_yaml' 2026-05-11T15:53:36,658 adding 'lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_driving_license.yaml' 2026-05-11T15:53:36,659 adding 'lm_eval/tasks/include/few_shot_en/Nepali/include_base_44_nepali_professional_certification.yaml' 2026-05-11T15:53:36,661 adding 'lm_eval/tasks/include/few_shot_en/Nepali/utils.py' 2026-05-11T15:53:36,662 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/_include_base_44_north macedonian.yaml' 2026-05-11T15:53:36,664 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/_north macedonian_few_shot_en_template_yaml' 2026-05-11T15:53:36,665 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml' 2026-05-11T15:53:36,666 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_business_commerce.yaml' 2026-05-11T15:53:36,668 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_social_science.yaml' 2026-05-11T15:53:36,669 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/include_base_44_north macedonian_stem.yaml' 2026-05-11T15:53:36,670 adding 'lm_eval/tasks/include/few_shot_en/North Macedonian/utils.py' 2026-05-11T15:53:36,672 adding 'lm_eval/tasks/include/few_shot_en/Persian/_include_base_44_persian.yaml' 2026-05-11T15:53:36,673 adding 'lm_eval/tasks/include/few_shot_en/Persian/_persian_few_shot_en_template_yaml' 2026-05-11T15:53:36,674 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_arts_humanities.yaml' 2026-05-11T15:53:36,675 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_driving_license.yaml' 2026-05-11T15:53:36,676 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_professional_certification.yaml' 2026-05-11T15:53:36,677 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_social_science.yaml' 2026-05-11T15:53:36,679 adding 'lm_eval/tasks/include/few_shot_en/Persian/include_base_44_persian_stem.yaml' 2026-05-11T15:53:36,680 adding 'lm_eval/tasks/include/few_shot_en/Persian/utils.py' 2026-05-11T15:53:36,682 adding 'lm_eval/tasks/include/few_shot_en/Polish/_include_base_44_polish.yaml' 2026-05-11T15:53:36,683 adding 'lm_eval/tasks/include/few_shot_en/Polish/_polish_few_shot_en_template_yaml' 2026-05-11T15:53:36,684 adding 'lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_professional_certification.yaml' 2026-05-11T15:53:36,685 adding 'lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_social_science.yaml' 2026-05-11T15:53:36,687 adding 'lm_eval/tasks/include/few_shot_en/Polish/include_base_44_polish_stem.yaml' 2026-05-11T15:53:36,688 adding 'lm_eval/tasks/include/few_shot_en/Polish/utils.py' 2026-05-11T15:53:36,690 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/_include_base_44_portuguese.yaml' 2026-05-11T15:53:36,691 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/_portuguese_few_shot_en_template_yaml' 2026-05-11T15:53:36,692 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_applied_science.yaml' 2026-05-11T15:53:36,693 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_arts_humanities.yaml' 2026-05-11T15:53:36,694 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_business_commerce.yaml' 2026-05-11T15:53:36,696 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_health_oriented_education.yaml' 2026-05-11T15:53:36,697 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_social_science.yaml' 2026-05-11T15:53:36,698 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/include_base_44_portuguese_stem.yaml' 2026-05-11T15:53:36,699 adding 'lm_eval/tasks/include/few_shot_en/Portuguese/utils.py' 2026-05-11T15:53:36,701 adding 'lm_eval/tasks/include/few_shot_en/Russian/_include_base_44_russian.yaml' 2026-05-11T15:53:36,702 adding 'lm_eval/tasks/include/few_shot_en/Russian/_russian_few_shot_en_template_yaml' 2026-05-11T15:53:36,703 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_applied_science.yaml' 2026-05-11T15:53:36,705 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_arts_humanities.yaml' 2026-05-11T15:53:36,706 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_business_commerce.yaml' 2026-05-11T15:53:36,707 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_driving_license.yaml' 2026-05-11T15:53:36,708 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_health_oriented_education.yaml' 2026-05-11T15:53:36,709 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_marine_license.yaml' 2026-05-11T15:53:36,710 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_social_science.yaml' 2026-05-11T15:53:36,712 adding 'lm_eval/tasks/include/few_shot_en/Russian/include_base_44_russian_stem.yaml' 2026-05-11T15:53:36,713 adding 'lm_eval/tasks/include/few_shot_en/Russian/utils.py' 2026-05-11T15:53:36,714 adding 'lm_eval/tasks/include/few_shot_en/Serbian/_include_base_44_serbian.yaml' 2026-05-11T15:53:36,715 adding 'lm_eval/tasks/include/few_shot_en/Serbian/_serbian_few_shot_en_template_yaml' 2026-05-11T15:53:36,717 adding 'lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_arts_humanities.yaml' 2026-05-11T15:53:36,718 adding 'lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_social_science.yaml' 2026-05-11T15:53:36,719 adding 'lm_eval/tasks/include/few_shot_en/Serbian/include_base_44_serbian_stem.yaml' 2026-05-11T15:53:36,720 adding 'lm_eval/tasks/include/few_shot_en/Serbian/utils.py' 2026-05-11T15:53:36,721 adding 'lm_eval/tasks/include/few_shot_en/Spanish/_include_base_44_spanish.yaml' 2026-05-11T15:53:36,723 adding 'lm_eval/tasks/include/few_shot_en/Spanish/_spanish_few_shot_en_template_yaml' 2026-05-11T15:53:36,724 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_arts_humanities.yaml' 2026-05-11T15:53:36,725 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_health_oriented_education.yaml' 2026-05-11T15:53:36,726 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_social_science.yaml' 2026-05-11T15:53:36,727 adding 'lm_eval/tasks/include/few_shot_en/Spanish/include_base_44_spanish_stem.yaml' 2026-05-11T15:53:36,729 adding 'lm_eval/tasks/include/few_shot_en/Spanish/utils.py' 2026-05-11T15:53:36,730 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/_include_base_44_tagalog.yaml' 2026-05-11T15:53:36,731 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/_tagalog_few_shot_en_template_yaml' 2026-05-11T15:53:36,733 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_arts_humanities.yaml' 2026-05-11T15:53:36,734 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/include_base_44_tagalog_driving_license.yaml' 2026-05-11T15:53:36,735 adding 'lm_eval/tasks/include/few_shot_en/Tagalog/utils.py' 2026-05-11T15:53:36,737 adding 'lm_eval/tasks/include/few_shot_en/Tamil/_include_base_44_tamil.yaml' 2026-05-11T15:53:36,738 adding 'lm_eval/tasks/include/few_shot_en/Tamil/_tamil_few_shot_en_template_yaml' 2026-05-11T15:53:36,739 adding 'lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_general_knowledge.yaml' 2026-05-11T15:53:36,740 adding 'lm_eval/tasks/include/few_shot_en/Tamil/include_base_44_tamil_stem.yaml' 2026-05-11T15:53:36,742 adding 'lm_eval/tasks/include/few_shot_en/Tamil/utils.py' 2026-05-11T15:53:36,743 adding 'lm_eval/tasks/include/few_shot_en/Telugu/_include_base_44_telugu.yaml' 2026-05-11T15:53:36,745 adding 'lm_eval/tasks/include/few_shot_en/Telugu/_telugu_few_shot_en_template_yaml' 2026-05-11T15:53:36,746 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_applied_science.yaml' 2026-05-11T15:53:36,747 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_arts_humanities.yaml' 2026-05-11T15:53:36,748 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_social_science.yaml' 2026-05-11T15:53:36,749 adding 'lm_eval/tasks/include/few_shot_en/Telugu/include_base_44_telugu_stem.yaml' 2026-05-11T15:53:36,750 adding 'lm_eval/tasks/include/few_shot_en/Telugu/utils.py' 2026-05-11T15:53:36,752 adding 'lm_eval/tasks/include/few_shot_en/Turkish/_include_base_44_turkish.yaml' 2026-05-11T15:53:36,753 adding 'lm_eval/tasks/include/few_shot_en/Turkish/_turkish_few_shot_en_template_yaml' 2026-05-11T15:53:36,755 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_arts_humanities.yaml' 2026-05-11T15:53:36,756 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_business_commerce.yaml' 2026-05-11T15:53:36,757 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_social_science.yaml' 2026-05-11T15:53:36,759 adding 'lm_eval/tasks/include/few_shot_en/Turkish/include_base_44_turkish_stem.yaml' 2026-05-11T15:53:36,760 adding 'lm_eval/tasks/include/few_shot_en/Turkish/utils.py' 2026-05-11T15:53:36,761 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/_include_base_44_ukrainian.yaml' 2026-05-11T15:53:36,762 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/_ukrainian_few_shot_en_template_yaml' 2026-05-11T15:53:36,764 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml' 2026-05-11T15:53:36,765 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_social_science.yaml' 2026-05-11T15:53:36,766 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/include_base_44_ukrainian_stem.yaml' 2026-05-11T15:53:36,767 adding 'lm_eval/tasks/include/few_shot_en/Ukrainian/utils.py' 2026-05-11T15:53:36,769 adding 'lm_eval/tasks/include/few_shot_en/Urdu/_include_base_44_urdu.yaml' 2026-05-11T15:53:36,770 adding 'lm_eval/tasks/include/few_shot_en/Urdu/_urdu_few_shot_en_template_yaml' 2026-05-11T15:53:36,771 adding 'lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_arts_humanities.yaml' 2026-05-11T15:53:36,772 adding 'lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_health_oriented_education.yaml' 2026-05-11T15:53:36,773 adding 'lm_eval/tasks/include/few_shot_en/Urdu/include_base_44_urdu_stem.yaml' 2026-05-11T15:53:36,774 adding 'lm_eval/tasks/include/few_shot_en/Urdu/utils.py' 2026-05-11T15:53:36,776 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/_include_base_44_uzbek.yaml' 2026-05-11T15:53:36,777 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/_uzbek_few_shot_en_template_yaml' 2026-05-11T15:53:36,779 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_arts_humanities.yaml' 2026-05-11T15:53:36,780 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_medical_license.yaml' 2026-05-11T15:53:36,781 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_social_science.yaml' 2026-05-11T15:53:36,782 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/include_base_44_uzbek_stem.yaml' 2026-05-11T15:53:36,783 adding 'lm_eval/tasks/include/few_shot_en/Uzbek/utils.py' 2026-05-11T15:53:36,785 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/_include_base_44_vietnamese.yaml' 2026-05-11T15:53:36,786 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/_vietnamese_few_shot_en_template_yaml' 2026-05-11T15:53:36,788 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml' 2026-05-11T15:53:36,789 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_social_science.yaml' 2026-05-11T15:53:36,790 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/include_base_44_vietnamese_stem.yaml' 2026-05-11T15:53:36,791 adding 'lm_eval/tasks/include/few_shot_en/Vietnamese/utils.py' 2026-05-11T15:53:36,794 adding 'lm_eval/tasks/include/few_shot_og/Albanian/_albanian_few_shot_og_template_yaml' 2026-05-11T15:53:36,795 adding 'lm_eval/tasks/include/few_shot_og/Albanian/_include_base_44_albanian.yaml' 2026-05-11T15:53:36,797 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_arts_humanities.yaml' 2026-05-11T15:53:36,798 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_business_commerce.yaml' 2026-05-11T15:53:36,799 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_health_oriented_education.yaml' 2026-05-11T15:53:36,800 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_social_science.yaml' 2026-05-11T15:53:36,802 adding 'lm_eval/tasks/include/few_shot_og/Albanian/include_base_44_albanian_stem.yaml' 2026-05-11T15:53:36,803 adding 'lm_eval/tasks/include/few_shot_og/Albanian/utils.py' 2026-05-11T15:53:36,805 adding 'lm_eval/tasks/include/few_shot_og/Arabic/_arabic_few_shot_og_template_yaml' 2026-05-11T15:53:36,806 adding 'lm_eval/tasks/include/few_shot_og/Arabic/_include_base_44_arabic.yaml' 2026-05-11T15:53:36,807 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_arts_humanities.yaml' 2026-05-11T15:53:36,808 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_business_commerce.yaml' 2026-05-11T15:53:36,809 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_driving_license.yaml' 2026-05-11T15:53:36,810 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_general_knowledge.yaml' 2026-05-11T15:53:36,812 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_social_science.yaml' 2026-05-11T15:53:36,813 adding 'lm_eval/tasks/include/few_shot_og/Arabic/include_base_44_arabic_stem.yaml' 2026-05-11T15:53:36,814 adding 'lm_eval/tasks/include/few_shot_og/Arabic/utils.py' 2026-05-11T15:53:36,816 adding 'lm_eval/tasks/include/few_shot_og/Armenian/_armenian_few_shot_og_template_yaml' 2026-05-11T15:53:36,817 adding 'lm_eval/tasks/include/few_shot_og/Armenian/_include_base_44_armenian.yaml' 2026-05-11T15:53:36,818 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_arts_humanities.yaml' 2026-05-11T15:53:36,819 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_driving_license.yaml' 2026-05-11T15:53:36,820 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_social_science.yaml' 2026-05-11T15:53:36,821 adding 'lm_eval/tasks/include/few_shot_og/Armenian/include_base_44_armenian_stem.yaml' 2026-05-11T15:53:36,822 adding 'lm_eval/tasks/include/few_shot_og/Armenian/utils.py' 2026-05-11T15:53:36,824 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/_azerbaijani_few_shot_og_template_yaml' 2026-05-11T15:53:36,825 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/_include_base_44_azerbaijani.yaml' 2026-05-11T15:53:36,826 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_applied_science.yaml' 2026-05-11T15:53:36,828 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_arts_humanities.yaml' 2026-05-11T15:53:36,829 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_business_commerce.yaml' 2026-05-11T15:53:36,830 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_health_oriented_education.yaml' 2026-05-11T15:53:36,831 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_social_science.yaml' 2026-05-11T15:53:36,832 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/include_base_44_azerbaijani_stem.yaml' 2026-05-11T15:53:36,833 adding 'lm_eval/tasks/include/few_shot_og/Azerbaijani/utils.py' 2026-05-11T15:53:36,835 adding 'lm_eval/tasks/include/few_shot_og/Basque/_basque_few_shot_og_template_yaml' 2026-05-11T15:53:36,836 adding 'lm_eval/tasks/include/few_shot_og/Basque/_include_base_44_basque.yaml' 2026-05-11T15:53:36,838 adding 'lm_eval/tasks/include/few_shot_og/Basque/include_base_44_basque_professional_certification.yaml' 2026-05-11T15:53:36,839 adding 'lm_eval/tasks/include/few_shot_og/Basque/utils.py' 2026-05-11T15:53:36,841 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/_belarusian_few_shot_og_template_yaml' 2026-05-11T15:53:36,842 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/_include_base_44_belarusian.yaml' 2026-05-11T15:53:36,843 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_arts_humanities.yaml' 2026-05-11T15:53:36,844 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_social_science.yaml' 2026-05-11T15:53:36,846 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/include_base_44_belarusian_stem.yaml' 2026-05-11T15:53:36,847 adding 'lm_eval/tasks/include/few_shot_og/Belarusian/utils.py' 2026-05-11T15:53:36,849 adding 'lm_eval/tasks/include/few_shot_og/Bengali/_bengali_few_shot_og_template_yaml' 2026-05-11T15:53:36,850 adding 'lm_eval/tasks/include/few_shot_og/Bengali/_include_base_44_bengali.yaml' 2026-05-11T15:53:36,851 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_arts_humanities.yaml' 2026-05-11T15:53:36,852 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_general_knowledge.yaml' 2026-05-11T15:53:36,853 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_professional_certification.yaml' 2026-05-11T15:53:36,855 adding 'lm_eval/tasks/include/few_shot_og/Bengali/include_base_44_bengali_stem.yaml' 2026-05-11T15:53:36,856 adding 'lm_eval/tasks/include/few_shot_og/Bengali/utils.py' 2026-05-11T15:53:36,857 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/_bulgarian_few_shot_og_template_yaml' 2026-05-11T15:53:36,859 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/_include_base_44_bulgarian.yaml' 2026-05-11T15:53:36,860 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_arts_humanities.yaml' 2026-05-11T15:53:36,861 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_social_science.yaml' 2026-05-11T15:53:36,862 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/include_base_44_bulgarian_stem.yaml' 2026-05-11T15:53:36,863 adding 'lm_eval/tasks/include/few_shot_og/Bulgarian/utils.py' 2026-05-11T15:53:36,865 adding 'lm_eval/tasks/include/few_shot_og/Chinese/_chinese_few_shot_og_template_yaml' 2026-05-11T15:53:36,866 adding 'lm_eval/tasks/include/few_shot_og/Chinese/_include_base_44_chinese.yaml' 2026-05-11T15:53:36,867 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_applied_science.yaml' 2026-05-11T15:53:36,868 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_arts_humanities.yaml' 2026-05-11T15:53:36,870 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_business_commerce.yaml' 2026-05-11T15:53:36,871 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_driving_license.yaml' 2026-05-11T15:53:36,872 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_health_oriented_education.yaml' 2026-05-11T15:53:36,873 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_professional_certification.yaml' 2026-05-11T15:53:36,874 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_social_science.yaml' 2026-05-11T15:53:36,875 adding 'lm_eval/tasks/include/few_shot_og/Chinese/include_base_44_chinese_stem.yaml' 2026-05-11T15:53:36,877 adding 'lm_eval/tasks/include/few_shot_og/Chinese/utils.py' 2026-05-11T15:53:36,879 adding 'lm_eval/tasks/include/few_shot_og/Croatian/_croatian_few_shot_og_template_yaml' 2026-05-11T15:53:36,880 adding 'lm_eval/tasks/include/few_shot_og/Croatian/_include_base_44_croatian.yaml' 2026-05-11T15:53:36,881 adding 'lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_arts_humanities.yaml' 2026-05-11T15:53:36,882 adding 'lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_social_science.yaml' 2026-05-11T15:53:36,883 adding 'lm_eval/tasks/include/few_shot_og/Croatian/include_base_44_croatian_stem.yaml' 2026-05-11T15:53:36,885 adding 'lm_eval/tasks/include/few_shot_og/Croatian/utils.py' 2026-05-11T15:53:36,886 adding 'lm_eval/tasks/include/few_shot_og/Dutch/_dutch_few_shot_og_template_yaml' 2026-05-11T15:53:36,888 adding 'lm_eval/tasks/include/few_shot_og/Dutch/_include_base_44_dutch.yaml' 2026-05-11T15:53:36,889 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_applied_science.yaml' 2026-05-11T15:53:36,890 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_arts_humanities.yaml' 2026-05-11T15:53:36,891 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_health_oriented_education.yaml' 2026-05-11T15:53:36,892 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_social_science.yaml' 2026-05-11T15:53:36,894 adding 'lm_eval/tasks/include/few_shot_og/Dutch/include_base_44_dutch_stem.yaml' 2026-05-11T15:53:36,895 adding 'lm_eval/tasks/include/few_shot_og/Dutch/utils.py' 2026-05-11T15:53:36,897 adding 'lm_eval/tasks/include/few_shot_og/Estonian/_estonian_few_shot_og_template_yaml' 2026-05-11T15:53:36,898 adding 'lm_eval/tasks/include/few_shot_og/Estonian/_include_base_44_estonian.yaml' 2026-05-11T15:53:36,899 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_applied_science.yaml' 2026-05-11T15:53:36,900 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_arts_humanities.yaml' 2026-05-11T15:53:36,901 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_health_oriented_education.yaml' 2026-05-11T15:53:36,902 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_social_science.yaml' 2026-05-11T15:53:36,903 adding 'lm_eval/tasks/include/few_shot_og/Estonian/include_base_44_estonian_stem.yaml' 2026-05-11T15:53:36,904 adding 'lm_eval/tasks/include/few_shot_og/Estonian/utils.py' 2026-05-11T15:53:36,906 adding 'lm_eval/tasks/include/few_shot_og/Finnish/_finnish_few_shot_og_template_yaml' 2026-05-11T15:53:36,907 adding 'lm_eval/tasks/include/few_shot_og/Finnish/_include_base_44_finnish.yaml' 2026-05-11T15:53:36,909 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_applied_science.yaml' 2026-05-11T15:53:36,910 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_arts_humanities.yaml' 2026-05-11T15:53:36,911 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_health_oriented_education.yaml' 2026-05-11T15:53:36,912 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_social_science.yaml' 2026-05-11T15:53:36,913 adding 'lm_eval/tasks/include/few_shot_og/Finnish/include_base_44_finnish_stem.yaml' 2026-05-11T15:53:36,914 adding 'lm_eval/tasks/include/few_shot_og/Finnish/utils.py' 2026-05-11T15:53:36,916 adding 'lm_eval/tasks/include/few_shot_og/French/_french_few_shot_og_template_yaml' 2026-05-11T15:53:36,917 adding 'lm_eval/tasks/include/few_shot_og/French/_include_base_44_french.yaml' 2026-05-11T15:53:36,918 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_arts_humanities.yaml' 2026-05-11T15:53:36,920 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_driving_license.yaml' 2026-05-11T15:53:36,921 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_health_oriented_education.yaml' 2026-05-11T15:53:36,922 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_social_science.yaml' 2026-05-11T15:53:36,923 adding 'lm_eval/tasks/include/few_shot_og/French/include_base_44_french_stem.yaml' 2026-05-11T15:53:36,924 adding 'lm_eval/tasks/include/few_shot_og/French/utils.py' 2026-05-11T15:53:36,926 adding 'lm_eval/tasks/include/few_shot_og/Georgian/_georgian_few_shot_og_template_yaml' 2026-05-11T15:53:36,927 adding 'lm_eval/tasks/include/few_shot_og/Georgian/_include_base_44_georgian.yaml' 2026-05-11T15:53:36,929 adding 'lm_eval/tasks/include/few_shot_og/Georgian/include_base_44_georgian_arts_humanities.yaml' 2026-05-11T15:53:36,930 adding 'lm_eval/tasks/include/few_shot_og/Georgian/utils.py' 2026-05-11T15:53:36,932 adding 'lm_eval/tasks/include/few_shot_og/German/_german_few_shot_og_template_yaml' 2026-05-11T15:53:36,933 adding 'lm_eval/tasks/include/few_shot_og/German/_include_base_44_german.yaml' 2026-05-11T15:53:36,934 adding 'lm_eval/tasks/include/few_shot_og/German/include_base_44_german_driving_license.yaml' 2026-05-11T15:53:36,935 adding 'lm_eval/tasks/include/few_shot_og/German/include_base_44_german_social_science.yaml' 2026-05-11T15:53:36,937 adding 'lm_eval/tasks/include/few_shot_og/German/include_base_44_german_stem.yaml' 2026-05-11T15:53:36,938 adding 'lm_eval/tasks/include/few_shot_og/German/utils.py' 2026-05-11T15:53:36,940 adding 'lm_eval/tasks/include/few_shot_og/Greek/_greek_few_shot_og_template_yaml' 2026-05-11T15:53:36,941 adding 'lm_eval/tasks/include/few_shot_og/Greek/_include_base_44_greek.yaml' 2026-05-11T15:53:36,943 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_arts_humanities.yaml' 2026-05-11T15:53:36,944 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_business_commerce.yaml' 2026-05-11T15:53:36,945 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_health_oriented_education.yaml' 2026-05-11T15:53:36,946 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_medical_license.yaml' 2026-05-11T15:53:36,947 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_professional_certification.yaml' 2026-05-11T15:53:36,948 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_social_science.yaml' 2026-05-11T15:53:36,950 adding 'lm_eval/tasks/include/few_shot_og/Greek/include_base_44_greek_stem.yaml' 2026-05-11T15:53:36,951 adding 'lm_eval/tasks/include/few_shot_og/Greek/utils.py' 2026-05-11T15:53:36,953 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/_hebrew_few_shot_og_template_yaml' 2026-05-11T15:53:36,954 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/_include_base_44_hebrew.yaml' 2026-05-11T15:53:36,955 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_arts_humanities.yaml' 2026-05-11T15:53:36,956 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/include_base_44_hebrew_driving_license.yaml' 2026-05-11T15:53:36,958 adding 'lm_eval/tasks/include/few_shot_og/Hebrew/utils.py' 2026-05-11T15:53:36,959 adding 'lm_eval/tasks/include/few_shot_og/Hindi/_hindi_few_shot_og_template_yaml' 2026-05-11T15:53:36,961 adding 'lm_eval/tasks/include/few_shot_og/Hindi/_include_base_44_hindi.yaml' 2026-05-11T15:53:36,962 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_applied_science.yaml' 2026-05-11T15:53:36,963 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_arts_humanities.yaml' 2026-05-11T15:53:36,964 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_driving_license.yaml' 2026-05-11T15:53:36,965 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_general_knowledge.yaml' 2026-05-11T15:53:36,967 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_health_oriented_education.yaml' 2026-05-11T15:53:36,968 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_professional_certification.yaml' 2026-05-11T15:53:36,969 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_social_science.yaml' 2026-05-11T15:53:36,970 adding 'lm_eval/tasks/include/few_shot_og/Hindi/include_base_44_hindi_stem.yaml' 2026-05-11T15:53:36,971 adding 'lm_eval/tasks/include/few_shot_og/Hindi/utils.py' 2026-05-11T15:53:36,973 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/_hungarian_few_shot_og_template_yaml' 2026-05-11T15:53:36,974 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/_include_base_44_hungarian.yaml' 2026-05-11T15:53:36,976 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_applied_science.yaml' 2026-05-11T15:53:36,977 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_social_science.yaml' 2026-05-11T15:53:36,978 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/include_base_44_hungarian_stem.yaml' 2026-05-11T15:53:36,979 adding 'lm_eval/tasks/include/few_shot_og/Hungarian/utils.py' 2026-05-11T15:53:36,981 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/_include_base_44_indonesian.yaml' 2026-05-11T15:53:36,982 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/_indonesian_few_shot_og_template_yaml' 2026-05-11T15:53:36,983 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_applied_science.yaml' 2026-05-11T15:53:36,985 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_arts_humanities.yaml' 2026-05-11T15:53:36,986 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_professional_certification.yaml' 2026-05-11T15:53:36,987 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_social_science.yaml' 2026-05-11T15:53:36,988 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/include_base_44_indonesian_stem.yaml' 2026-05-11T15:53:36,990 adding 'lm_eval/tasks/include/few_shot_og/Indonesian/utils.py' 2026-05-11T15:53:36,991 adding 'lm_eval/tasks/include/few_shot_og/Italian/_include_base_44_italian.yaml' 2026-05-11T15:53:36,992 adding 'lm_eval/tasks/include/few_shot_og/Italian/_italian_few_shot_og_template_yaml' 2026-05-11T15:53:36,993 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_applied_science.yaml' 2026-05-11T15:53:36,995 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_arts_humanities.yaml' 2026-05-11T15:53:36,996 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_health_oriented_education.yaml' 2026-05-11T15:53:36,997 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_professional_certification.yaml' 2026-05-11T15:53:36,998 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_social_science.yaml' 2026-05-11T15:53:36,999 adding 'lm_eval/tasks/include/few_shot_og/Italian/include_base_44_italian_stem.yaml' 2026-05-11T15:53:37,000 adding 'lm_eval/tasks/include/few_shot_og/Italian/utils.py' 2026-05-11T15:53:37,002 adding 'lm_eval/tasks/include/few_shot_og/Japanese/_include_base_44_japanese.yaml' 2026-05-11T15:53:37,003 adding 'lm_eval/tasks/include/few_shot_og/Japanese/_japanese_few_shot_og_template_yaml' 2026-05-11T15:53:37,004 adding 'lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_driving_license.yaml' 2026-05-11T15:53:37,005 adding 'lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_medical_license.yaml' 2026-05-11T15:53:37,006 adding 'lm_eval/tasks/include/few_shot_og/Japanese/include_base_44_japanese_professional_certification.yaml' 2026-05-11T15:53:37,008 adding 'lm_eval/tasks/include/few_shot_og/Japanese/utils.py' 2026-05-11T15:53:37,009 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/_include_base_44_kazakh.yaml' 2026-05-11T15:53:37,011 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/_kazakh_few_shot_og_template_yaml' 2026-05-11T15:53:37,012 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/include_base_44_kazakh_arts_humanities.yaml' 2026-05-11T15:53:37,013 adding 'lm_eval/tasks/include/few_shot_og/Kazakh/utils.py' 2026-05-11T15:53:37,015 adding 'lm_eval/tasks/include/few_shot_og/Korean/_include_base_44_korean.yaml' 2026-05-11T15:53:37,016 adding 'lm_eval/tasks/include/few_shot_og/Korean/_korean_few_shot_og_template_yaml' 2026-05-11T15:53:37,017 adding 'lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_professional_certification.yaml' 2026-05-11T15:53:37,018 adding 'lm_eval/tasks/include/few_shot_og/Korean/include_base_44_korean_social_science.yaml' 2026-05-11T15:53:37,019 adding 'lm_eval/tasks/include/few_shot_og/Korean/utils.py' 2026-05-11T15:53:37,021 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/_include_base_44_lithuanian.yaml' 2026-05-11T15:53:37,022 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/_lithuanian_few_shot_og_template_yaml' 2026-05-11T15:53:37,024 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_arts_humanities.yaml' 2026-05-11T15:53:37,025 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_business_commerce.yaml' 2026-05-11T15:53:37,026 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_professional_certification.yaml' 2026-05-11T15:53:37,027 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_social_science.yaml' 2026-05-11T15:53:37,028 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/include_base_44_lithuanian_stem.yaml' 2026-05-11T15:53:37,030 adding 'lm_eval/tasks/include/few_shot_og/Lithuanian/utils.py' 2026-05-11T15:53:37,031 adding 'lm_eval/tasks/include/few_shot_og/Malay/_include_base_44_malay.yaml' 2026-05-11T15:53:37,033 adding 'lm_eval/tasks/include/few_shot_og/Malay/_malay_few_shot_og_template_yaml' 2026-05-11T15:53:37,034 adding 'lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_arts_humanities.yaml' 2026-05-11T15:53:37,035 adding 'lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_business_commerce.yaml' 2026-05-11T15:53:37,036 adding 'lm_eval/tasks/include/few_shot_og/Malay/include_base_44_malay_social_science.yaml' 2026-05-11T15:53:37,037 adding 'lm_eval/tasks/include/few_shot_og/Malay/utils.py' 2026-05-11T15:53:37,039 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/_include_base_44_malayalam.yaml' 2026-05-11T15:53:37,040 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/_malayalam_few_shot_og_template_yaml' 2026-05-11T15:53:37,041 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_arts_humanities.yaml' 2026-05-11T15:53:37,042 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_general_knowledge.yaml' 2026-05-11T15:53:37,044 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_health_oriented_education.yaml' 2026-05-11T15:53:37,045 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_marine_license.yaml' 2026-05-11T15:53:37,046 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_social_science.yaml' 2026-05-11T15:53:37,047 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/include_base_44_malayalam_stem.yaml' 2026-05-11T15:53:37,049 adding 'lm_eval/tasks/include/few_shot_og/Malayalam/utils.py' 2026-05-11T15:53:37,050 adding 'lm_eval/tasks/include/few_shot_og/Nepali/_include_base_44_nepali.yaml' 2026-05-11T15:53:37,052 adding 'lm_eval/tasks/include/few_shot_og/Nepali/_nepali_few_shot_og_template_yaml' 2026-05-11T15:53:37,053 adding 'lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_driving_license.yaml' 2026-05-11T15:53:37,054 adding 'lm_eval/tasks/include/few_shot_og/Nepali/include_base_44_nepali_professional_certification.yaml' 2026-05-11T15:53:37,055 adding 'lm_eval/tasks/include/few_shot_og/Nepali/utils.py' 2026-05-11T15:53:37,057 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/_include_base_44_north macedonian.yaml' 2026-05-11T15:53:37,058 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/_north macedonian_few_shot_og_template_yaml' 2026-05-11T15:53:37,059 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_arts_humanities.yaml' 2026-05-11T15:53:37,060 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_business_commerce.yaml' 2026-05-11T15:53:37,062 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_social_science.yaml' 2026-05-11T15:53:37,063 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/include_base_44_north macedonian_stem.yaml' 2026-05-11T15:53:37,064 adding 'lm_eval/tasks/include/few_shot_og/North Macedonian/utils.py' 2026-05-11T15:53:37,066 adding 'lm_eval/tasks/include/few_shot_og/Persian/_include_base_44_persian.yaml' 2026-05-11T15:53:37,067 adding 'lm_eval/tasks/include/few_shot_og/Persian/_persian_few_shot_og_template_yaml' 2026-05-11T15:53:37,068 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_arts_humanities.yaml' 2026-05-11T15:53:37,069 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_driving_license.yaml' 2026-05-11T15:53:37,070 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_professional_certification.yaml' 2026-05-11T15:53:37,072 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_social_science.yaml' 2026-05-11T15:53:37,073 adding 'lm_eval/tasks/include/few_shot_og/Persian/include_base_44_persian_stem.yaml' 2026-05-11T15:53:37,074 adding 'lm_eval/tasks/include/few_shot_og/Persian/utils.py' 2026-05-11T15:53:37,076 adding 'lm_eval/tasks/include/few_shot_og/Polish/_include_base_44_polish.yaml' 2026-05-11T15:53:37,077 adding 'lm_eval/tasks/include/few_shot_og/Polish/_polish_few_shot_og_template_yaml' 2026-05-11T15:53:37,078 adding 'lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_professional_certification.yaml' 2026-05-11T15:53:37,080 adding 'lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_social_science.yaml' 2026-05-11T15:53:37,081 adding 'lm_eval/tasks/include/few_shot_og/Polish/include_base_44_polish_stem.yaml' 2026-05-11T15:53:37,082 adding 'lm_eval/tasks/include/few_shot_og/Polish/utils.py' 2026-05-11T15:53:37,084 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/_include_base_44_portuguese.yaml' 2026-05-11T15:53:37,085 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/_portuguese_few_shot_og_template_yaml' 2026-05-11T15:53:37,087 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_applied_science.yaml' 2026-05-11T15:53:37,088 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_arts_humanities.yaml' 2026-05-11T15:53:37,089 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_business_commerce.yaml' 2026-05-11T15:53:37,091 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_health_oriented_education.yaml' 2026-05-11T15:53:37,092 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_social_science.yaml' 2026-05-11T15:53:37,093 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/include_base_44_portuguese_stem.yaml' 2026-05-11T15:53:37,095 adding 'lm_eval/tasks/include/few_shot_og/Portuguese/utils.py' 2026-05-11T15:53:37,096 adding 'lm_eval/tasks/include/few_shot_og/Russian/_include_base_44_russian.yaml' 2026-05-11T15:53:37,097 adding 'lm_eval/tasks/include/few_shot_og/Russian/_russian_few_shot_og_template_yaml' 2026-05-11T15:53:37,099 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_applied_science.yaml' 2026-05-11T15:53:37,100 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_arts_humanities.yaml' 2026-05-11T15:53:37,101 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_business_commerce.yaml' 2026-05-11T15:53:37,103 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_driving_license.yaml' 2026-05-11T15:53:37,104 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_health_oriented_education.yaml' 2026-05-11T15:53:37,105 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_marine_license.yaml' 2026-05-11T15:53:37,106 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_social_science.yaml' 2026-05-11T15:53:37,107 adding 'lm_eval/tasks/include/few_shot_og/Russian/include_base_44_russian_stem.yaml' 2026-05-11T15:53:37,109 adding 'lm_eval/tasks/include/few_shot_og/Russian/utils.py' 2026-05-11T15:53:37,111 adding 'lm_eval/tasks/include/few_shot_og/Serbian/_include_base_44_serbian.yaml' 2026-05-11T15:53:37,112 adding 'lm_eval/tasks/include/few_shot_og/Serbian/_serbian_few_shot_og_template_yaml' 2026-05-11T15:53:37,113 adding 'lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_arts_humanities.yaml' 2026-05-11T15:53:37,114 adding 'lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_social_science.yaml' 2026-05-11T15:53:37,115 adding 'lm_eval/tasks/include/few_shot_og/Serbian/include_base_44_serbian_stem.yaml' 2026-05-11T15:53:37,117 adding 'lm_eval/tasks/include/few_shot_og/Serbian/utils.py' 2026-05-11T15:53:37,119 adding 'lm_eval/tasks/include/few_shot_og/Spanish/_include_base_44_spanish.yaml' 2026-05-11T15:53:37,120 adding 'lm_eval/tasks/include/few_shot_og/Spanish/_spanish_few_shot_og_template_yaml' 2026-05-11T15:53:37,121 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_arts_humanities.yaml' 2026-05-11T15:53:37,122 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_health_oriented_education.yaml' 2026-05-11T15:53:37,124 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_social_science.yaml' 2026-05-11T15:53:37,125 adding 'lm_eval/tasks/include/few_shot_og/Spanish/include_base_44_spanish_stem.yaml' 2026-05-11T15:53:37,126 adding 'lm_eval/tasks/include/few_shot_og/Spanish/utils.py' 2026-05-11T15:53:37,128 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/_include_base_44_tagalog.yaml' 2026-05-11T15:53:37,129 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/_tagalog_few_shot_og_template_yaml' 2026-05-11T15:53:37,130 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_arts_humanities.yaml' 2026-05-11T15:53:37,131 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/include_base_44_tagalog_driving_license.yaml' 2026-05-11T15:53:37,132 adding 'lm_eval/tasks/include/few_shot_og/Tagalog/utils.py' 2026-05-11T15:53:37,134 adding 'lm_eval/tasks/include/few_shot_og/Tamil/_include_base_44_tamil.yaml' 2026-05-11T15:53:37,135 adding 'lm_eval/tasks/include/few_shot_og/Tamil/_tamil_few_shot_og_template_yaml' 2026-05-11T15:53:37,137 adding 'lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_general_knowledge.yaml' 2026-05-11T15:53:37,138 adding 'lm_eval/tasks/include/few_shot_og/Tamil/include_base_44_tamil_stem.yaml' 2026-05-11T15:53:37,139 adding 'lm_eval/tasks/include/few_shot_og/Tamil/utils.py' 2026-05-11T15:53:37,141 adding 'lm_eval/tasks/include/few_shot_og/Telugu/_include_base_44_telugu.yaml' 2026-05-11T15:53:37,142 adding 'lm_eval/tasks/include/few_shot_og/Telugu/_telugu_few_shot_og_template_yaml' 2026-05-11T15:53:37,143 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_applied_science.yaml' 2026-05-11T15:53:37,145 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_arts_humanities.yaml' 2026-05-11T15:53:37,146 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_social_science.yaml' 2026-05-11T15:53:37,147 adding 'lm_eval/tasks/include/few_shot_og/Telugu/include_base_44_telugu_stem.yaml' 2026-05-11T15:53:37,148 adding 'lm_eval/tasks/include/few_shot_og/Telugu/utils.py' 2026-05-11T15:53:37,150 adding 'lm_eval/tasks/include/few_shot_og/Turkish/_include_base_44_turkish.yaml' 2026-05-11T15:53:37,151 adding 'lm_eval/tasks/include/few_shot_og/Turkish/_turkish_few_shot_og_template_yaml' 2026-05-11T15:53:37,153 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_arts_humanities.yaml' 2026-05-11T15:53:37,154 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_business_commerce.yaml' 2026-05-11T15:53:37,155 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_social_science.yaml' 2026-05-11T15:53:37,156 adding 'lm_eval/tasks/include/few_shot_og/Turkish/include_base_44_turkish_stem.yaml' 2026-05-11T15:53:37,157 adding 'lm_eval/tasks/include/few_shot_og/Turkish/utils.py' 2026-05-11T15:53:37,159 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/_include_base_44_ukrainian.yaml' 2026-05-11T15:53:37,160 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/_ukrainian_few_shot_og_template_yaml' 2026-05-11T15:53:37,162 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_arts_humanities.yaml' 2026-05-11T15:53:37,163 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_social_science.yaml' 2026-05-11T15:53:37,164 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/include_base_44_ukrainian_stem.yaml' 2026-05-11T15:53:37,165 adding 'lm_eval/tasks/include/few_shot_og/Ukrainian/utils.py' 2026-05-11T15:53:37,167 adding 'lm_eval/tasks/include/few_shot_og/Urdu/_include_base_44_urdu.yaml' 2026-05-11T15:53:37,168 adding 'lm_eval/tasks/include/few_shot_og/Urdu/_urdu_few_shot_og_template_yaml' 2026-05-11T15:53:37,169 adding 'lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_arts_humanities.yaml' 2026-05-11T15:53:37,171 adding 'lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_health_oriented_education.yaml' 2026-05-11T15:53:37,172 adding 'lm_eval/tasks/include/few_shot_og/Urdu/include_base_44_urdu_stem.yaml' 2026-05-11T15:53:37,173 adding 'lm_eval/tasks/include/few_shot_og/Urdu/utils.py' 2026-05-11T15:53:37,175 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/_include_base_44_uzbek.yaml' 2026-05-11T15:53:37,176 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/_uzbek_few_shot_og_template_yaml' 2026-05-11T15:53:37,177 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_arts_humanities.yaml' 2026-05-11T15:53:37,178 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_medical_license.yaml' 2026-05-11T15:53:37,179 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_social_science.yaml' 2026-05-11T15:53:37,180 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/include_base_44_uzbek_stem.yaml' 2026-05-11T15:53:37,182 adding 'lm_eval/tasks/include/few_shot_og/Uzbek/utils.py' 2026-05-11T15:53:37,183 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/_include_base_44_vietnamese.yaml' 2026-05-11T15:53:37,184 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/_vietnamese_few_shot_og_template_yaml' 2026-05-11T15:53:37,186 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_arts_humanities.yaml' 2026-05-11T15:53:37,187 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_social_science.yaml' 2026-05-11T15:53:37,188 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/include_base_44_vietnamese_stem.yaml' 2026-05-11T15:53:37,189 adding 'lm_eval/tasks/include/few_shot_og/Vietnamese/utils.py' 2026-05-11T15:53:37,191 adding 'lm_eval/tasks/inverse_scaling/README.md' 2026-05-11T15:53:37,192 adding 'lm_eval/tasks/inverse_scaling/_inverse_scaling_mc_yaml' 2026-05-11T15:53:37,194 adding 'lm_eval/tasks/inverse_scaling/_some_results' 2026-05-11T15:53:37,195 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_hindsight_neglect.yaml' 2026-05-11T15:53:37,196 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_into_the_unknown.yaml' 2026-05-11T15:53:37,197 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_memo_trap.yaml' 2026-05-11T15:53:37,198 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_modus_tollens.yaml' 2026-05-11T15:53:37,199 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_neqa.yaml' 2026-05-11T15:53:37,201 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_pattern_matching_suppression.yaml' 2026-05-11T15:53:37,202 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_quote_repetition.yaml' 2026-05-11T15:53:37,203 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_redefine_math.yaml' 2026-05-11T15:53:37,204 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_repetitive_algebra.yaml' 2026-05-11T15:53:37,205 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_sig_figs.yaml' 2026-05-11T15:53:37,206 adding 'lm_eval/tasks/inverse_scaling/inverse_scaling_winobias_antistereotype.yaml' 2026-05-11T15:53:37,209 adding 'lm_eval/tasks/japanese_leaderboard/README.md' 2026-05-11T15:53:37,210 adding 'lm_eval/tasks/japanese_leaderboard/_ja_leaderboard.yaml' 2026-05-11T15:53:37,211 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jaqket_v2.yaml' 2026-05-11T15:53:37,212 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.py' 2026-05-11T15:53:37,214 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jcommonsenseqa.yaml' 2026-05-11T15:53:37,215 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jnli.yaml' 2026-05-11T15:53:37,216 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_jsquad.yaml' 2026-05-11T15:53:37,218 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_marc_ja.yaml' 2026-05-11T15:53:37,219 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.py' 2026-05-11T15:53:37,220 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_mgsm.yaml' 2026-05-11T15:53:37,221 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.py' 2026-05-11T15:53:37,223 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xlsum.yaml' 2026-05-11T15:53:37,224 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.py' 2026-05-11T15:53:37,225 adding 'lm_eval/tasks/japanese_leaderboard/ja_leaderboard_xwinograd.yaml' 2026-05-11T15:53:37,226 adding 'lm_eval/tasks/japanese_leaderboard/requirements.txt' 2026-05-11T15:53:37,229 adding 'lm_eval/tasks/jsonschema_bench/README.md' 2026-05-11T15:53:37,230 adding 'lm_eval/tasks/jsonschema_bench/jsonschema_bench_easy.yaml' 2026-05-11T15:53:37,231 adding 'lm_eval/tasks/jsonschema_bench/jsonschema_bench_hard.yaml' 2026-05-11T15:53:37,232 adding 'lm_eval/tasks/jsonschema_bench/jsonschema_bench_medium.yaml' 2026-05-11T15:53:37,234 adding 'lm_eval/tasks/jsonschema_bench/metrics.py' 2026-05-11T15:53:37,236 adding 'lm_eval/tasks/kbl/README.md' 2026-05-11T15:53:37,238 adding 'lm_eval/tasks/kbl/bar_exam/civil/_base_em_yaml' 2026-05-11T15:53:37,239 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2012.yaml' 2026-05-11T15:53:37,240 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2013.yaml' 2026-05-11T15:53:37,241 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2014.yaml' 2026-05-11T15:53:37,242 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2015.yaml' 2026-05-11T15:53:37,244 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2016.yaml' 2026-05-11T15:53:37,245 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2017.yaml' 2026-05-11T15:53:37,246 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2018.yaml' 2026-05-11T15:53:37,249 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2019.yaml' 2026-05-11T15:53:37,250 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2020.yaml' 2026-05-11T15:53:37,251 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2021.yaml' 2026-05-11T15:53:37,252 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2022.yaml' 2026-05-11T15:53:37,253 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2023.yaml' 2026-05-11T15:53:37,254 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2024.yaml' 2026-05-11T15:53:37,255 adding 'lm_eval/tasks/kbl/bar_exam/civil/kbl_bar_exam_em_civil_2025.yaml' 2026-05-11T15:53:37,257 adding 'lm_eval/tasks/kbl/bar_exam/criminal/_base_em_yaml' 2026-05-11T15:53:37,259 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2012.yaml' 2026-05-11T15:53:37,260 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2013.yaml' 2026-05-11T15:53:37,261 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2014.yaml' 2026-05-11T15:53:37,262 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2015.yaml' 2026-05-11T15:53:37,263 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2016.yaml' 2026-05-11T15:53:37,264 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2017.yaml' 2026-05-11T15:53:37,266 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2018.yaml' 2026-05-11T15:53:37,267 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2019.yaml' 2026-05-11T15:53:37,268 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2020.yaml' 2026-05-11T15:53:37,269 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2021.yaml' 2026-05-11T15:53:37,270 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2022.yaml' 2026-05-11T15:53:37,271 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2023.yaml' 2026-05-11T15:53:37,273 adding 'lm_eval/tasks/kbl/bar_exam/criminal/kbl_bar_exam_em_criminal_2024.yaml' 2026-05-11T15:53:37,275 adding 'lm_eval/tasks/kbl/bar_exam/public/_base_em_yaml' 2026-05-11T15:53:37,276 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2012.yaml' 2026-05-11T15:53:37,277 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2013.yaml' 2026-05-11T15:53:37,278 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2014.yaml' 2026-05-11T15:53:37,280 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2015.yaml' 2026-05-11T15:53:37,281 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2016.yaml' 2026-05-11T15:53:37,282 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2017.yaml' 2026-05-11T15:53:37,283 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2018.yaml' 2026-05-11T15:53:37,284 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2019.yaml' 2026-05-11T15:53:37,285 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2020.yaml' 2026-05-11T15:53:37,286 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2021.yaml' 2026-05-11T15:53:37,287 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2022.yaml' 2026-05-11T15:53:37,288 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2023.yaml' 2026-05-11T15:53:37,289 adding 'lm_eval/tasks/kbl/bar_exam/public/kbl_bar_exam_em_public_2024.yaml' 2026-05-11T15:53:37,291 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/_base_em_yaml' 2026-05-11T15:53:37,292 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2010.yaml' 2026-05-11T15:53:37,293 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2011.yaml' 2026-05-11T15:53:37,294 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2012.yaml' 2026-05-11T15:53:37,295 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2013.yaml' 2026-05-11T15:53:37,297 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2014.yaml' 2026-05-11T15:53:37,298 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2015.yaml' 2026-05-11T15:53:37,299 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2016.yaml' 2026-05-11T15:53:37,300 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2017.yaml' 2026-05-11T15:53:37,301 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2018.yaml' 2026-05-11T15:53:37,302 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2019.yaml' 2026-05-11T15:53:37,303 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2020.yaml' 2026-05-11T15:53:37,304 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2021.yaml' 2026-05-11T15:53:37,305 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2022.yaml' 2026-05-11T15:53:37,307 adding 'lm_eval/tasks/kbl/bar_exam/responsibility/kbl_bar_exam_em_responsibility_2023.yaml' 2026-05-11T15:53:37,308 adding 'lm_eval/tasks/kbl/knowledge/_kbl_knowledge_yaml' 2026-05-11T15:53:37,310 adding 'lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_em.yaml' 2026-05-11T15:53:37,311 adding 'lm_eval/tasks/kbl/knowledge/kbl_common_legal_mistake_qa_reasoning_em.yaml' 2026-05-11T15:53:37,312 adding 'lm_eval/tasks/kbl/knowledge/kbl_legal_concept_qa_em.yaml' 2026-05-11T15:53:37,313 adding 'lm_eval/tasks/kbl/knowledge/kbl_offense_component_qa_em.yaml' 2026-05-11T15:53:37,315 adding 'lm_eval/tasks/kbl/knowledge/kbl_query_statute_matching_qa_em.yaml' 2026-05-11T15:53:37,316 adding 'lm_eval/tasks/kbl/knowledge/kbl_statute_hallucination_qa_em.yaml' 2026-05-11T15:53:37,317 adding 'lm_eval/tasks/kbl/knowledge/kbl_statute_number_and_content_matching_qa_em.yaml' 2026-05-11T15:53:37,319 adding 'lm_eval/tasks/kbl/reasoning/_kbl_reasoning_yaml' 2026-05-11T15:53:37,320 adding 'lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_p_em.yaml' 2026-05-11T15:53:37,322 adding 'lm_eval/tasks/kbl/reasoning/kbl_case_relevance_qa_q_em.yaml' 2026-05-11T15:53:37,323 adding 'lm_eval/tasks/kbl/reasoning/kbl_causal_reasoning_em.yaml' 2026-05-11T15:53:37,324 adding 'lm_eval/tasks/kbl/reasoning/kbl_statement_consistency_qa_em.yaml' 2026-05-11T15:53:37,326 adding 'lm_eval/tasks/kmmlu/README.md' 2026-05-11T15:53:37,329 adding 'lm_eval/tasks/kmmlu/cot_hard/_cot_kmmlu_yaml' 2026-05-11T15:53:37,330 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard.yaml' 2026-05-11T15:53:37,331 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_applied_science.yaml' 2026-05-11T15:53:37,332 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_humss.yaml' 2026-05-11T15:53:37,333 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_other.yaml' 2026-05-11T15:53:37,334 adding 'lm_eval/tasks/kmmlu/cot_hard/_kmmlu_cot_hard_stem.yaml' 2026-05-11T15:53:37,336 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_accounting.yaml' 2026-05-11T15:53:37,338 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_agricultural_sciences.yaml' 2026-05-11T15:53:37,340 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_aviation_engineering_and_maintenance.yaml' 2026-05-11T15:53:37,341 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_biology.yaml' 2026-05-11T15:53:37,343 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemical_engineering.yaml' 2026-05-11T15:53:37,345 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_chemistry.yaml' 2026-05-11T15:53:37,347 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_civil_engineering.yaml' 2026-05-11T15:53:37,349 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_computer_science.yaml' 2026-05-11T15:53:37,351 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_construction.yaml' 2026-05-11T15:53:37,353 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_criminal_law.yaml' 2026-05-11T15:53:37,355 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_ecology.yaml' 2026-05-11T15:53:37,357 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_economics.yaml' 2026-05-11T15:53:37,359 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_education.yaml' 2026-05-11T15:53:37,362 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electrical_engineering.yaml' 2026-05-11T15:53:37,363 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_electronics_engineering.yaml' 2026-05-11T15:53:37,365 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_energy_management.yaml' 2026-05-11T15:53:37,367 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_environmental_science.yaml' 2026-05-11T15:53:37,369 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_fashion.yaml' 2026-05-11T15:53:37,371 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_food_processing.yaml' 2026-05-11T15:53:37,373 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_gas_technology_and_engineering.yaml' 2026-05-11T15:53:37,375 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_geomatics.yaml' 2026-05-11T15:53:37,376 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_health.yaml' 2026-05-11T15:53:37,378 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_industrial_engineer.yaml' 2026-05-11T15:53:37,380 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_information_technology.yaml' 2026-05-11T15:53:37,382 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_interior_architecture_and_design.yaml' 2026-05-11T15:53:37,384 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_korean_history.yaml' 2026-05-11T15:53:37,386 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_law.yaml' 2026-05-11T15:53:37,388 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_machine_design_and_manufacturing.yaml' 2026-05-11T15:53:37,390 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_management.yaml' 2026-05-11T15:53:37,392 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_maritime_engineering.yaml' 2026-05-11T15:53:37,394 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_marketing.yaml' 2026-05-11T15:53:37,396 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_materials_engineering.yaml' 2026-05-11T15:53:37,398 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_math.yaml' 2026-05-11T15:53:37,400 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_mechanical_engineering.yaml' 2026-05-11T15:53:37,402 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_nondestructive_testing.yaml' 2026-05-11T15:53:37,404 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_patent.yaml' 2026-05-11T15:53:37,406 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_political_science_and_sociology.yaml' 2026-05-11T15:53:37,408 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_psychology.yaml' 2026-05-11T15:53:37,410 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_public_safety.yaml' 2026-05-11T15:53:37,412 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_railway_and_automotive_engineering.yaml' 2026-05-11T15:53:37,414 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_real_estate.yaml' 2026-05-11T15:53:37,416 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_refrigerating_machinery.yaml' 2026-05-11T15:53:37,418 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_social_welfare.yaml' 2026-05-11T15:53:37,420 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_taxation.yaml' 2026-05-11T15:53:37,422 adding 'lm_eval/tasks/kmmlu/cot_hard/kmmlu_cot_hard_telecommunications_and_wireless_technology.yaml' 2026-05-11T15:53:37,424 adding 'lm_eval/tasks/kmmlu/default/_default_kmmlu_yaml' 2026-05-11T15:53:37,425 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_applied_science.yaml' 2026-05-11T15:53:37,427 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_default.yaml' 2026-05-11T15:53:37,428 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_humss.yaml' 2026-05-11T15:53:37,429 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_other.yaml' 2026-05-11T15:53:37,430 adding 'lm_eval/tasks/kmmlu/default/_kmmlu_stem.yaml' 2026-05-11T15:53:37,431 adding 'lm_eval/tasks/kmmlu/default/kmmlu_accounting.yaml' 2026-05-11T15:53:37,432 adding 'lm_eval/tasks/kmmlu/default/kmmlu_agricultural_sciences.yaml' 2026-05-11T15:53:37,433 adding 'lm_eval/tasks/kmmlu/default/kmmlu_aviation_engineering_and_maintenance.yaml' 2026-05-11T15:53:37,434 adding 'lm_eval/tasks/kmmlu/default/kmmlu_biology.yaml' 2026-05-11T15:53:37,435 adding 'lm_eval/tasks/kmmlu/default/kmmlu_chemical_engineering.yaml' 2026-05-11T15:53:37,437 adding 'lm_eval/tasks/kmmlu/default/kmmlu_chemistry.yaml' 2026-05-11T15:53:37,438 adding 'lm_eval/tasks/kmmlu/default/kmmlu_civil_engineering.yaml' 2026-05-11T15:53:37,439 adding 'lm_eval/tasks/kmmlu/default/kmmlu_computer_science.yaml' 2026-05-11T15:53:37,440 adding 'lm_eval/tasks/kmmlu/default/kmmlu_construction.yaml' 2026-05-11T15:53:37,441 adding 'lm_eval/tasks/kmmlu/default/kmmlu_criminal_law.yaml' 2026-05-11T15:53:37,442 adding 'lm_eval/tasks/kmmlu/default/kmmlu_ecology.yaml' 2026-05-11T15:53:37,443 adding 'lm_eval/tasks/kmmlu/default/kmmlu_economics.yaml' 2026-05-11T15:53:37,445 adding 'lm_eval/tasks/kmmlu/default/kmmlu_education.yaml' 2026-05-11T15:53:37,446 adding 'lm_eval/tasks/kmmlu/default/kmmlu_electrical_engineering.yaml' 2026-05-11T15:53:37,447 adding 'lm_eval/tasks/kmmlu/default/kmmlu_electronics_engineering.yaml' 2026-05-11T15:53:37,448 adding 'lm_eval/tasks/kmmlu/default/kmmlu_energy_management.yaml' 2026-05-11T15:53:37,449 adding 'lm_eval/tasks/kmmlu/default/kmmlu_environmental_science.yaml' 2026-05-11T15:53:37,450 adding 'lm_eval/tasks/kmmlu/default/kmmlu_fashion.yaml' 2026-05-11T15:53:37,452 adding 'lm_eval/tasks/kmmlu/default/kmmlu_food_processing.yaml' 2026-05-11T15:53:37,453 adding 'lm_eval/tasks/kmmlu/default/kmmlu_gas_technology_and_engineering.yaml' 2026-05-11T15:53:37,454 adding 'lm_eval/tasks/kmmlu/default/kmmlu_geomatics.yaml' 2026-05-11T15:53:37,455 adding 'lm_eval/tasks/kmmlu/default/kmmlu_health.yaml' 2026-05-11T15:53:37,456 adding 'lm_eval/tasks/kmmlu/default/kmmlu_industrial_engineer.yaml' 2026-05-11T15:53:37,458 adding 'lm_eval/tasks/kmmlu/default/kmmlu_information_technology.yaml' 2026-05-11T15:53:37,459 adding 'lm_eval/tasks/kmmlu/default/kmmlu_interior_architecture_and_design.yaml' 2026-05-11T15:53:37,460 adding 'lm_eval/tasks/kmmlu/default/kmmlu_korean_history.yaml' 2026-05-11T15:53:37,461 adding 'lm_eval/tasks/kmmlu/default/kmmlu_law.yaml' 2026-05-11T15:53:37,462 adding 'lm_eval/tasks/kmmlu/default/kmmlu_machine_design_and_manufacturing.yaml' 2026-05-11T15:53:37,463 adding 'lm_eval/tasks/kmmlu/default/kmmlu_management.yaml' 2026-05-11T15:53:37,464 adding 'lm_eval/tasks/kmmlu/default/kmmlu_maritime_engineering.yaml' 2026-05-11T15:53:37,466 adding 'lm_eval/tasks/kmmlu/default/kmmlu_marketing.yaml' 2026-05-11T15:53:37,467 adding 'lm_eval/tasks/kmmlu/default/kmmlu_materials_engineering.yaml' 2026-05-11T15:53:37,468 adding 'lm_eval/tasks/kmmlu/default/kmmlu_math.yaml' 2026-05-11T15:53:37,469 adding 'lm_eval/tasks/kmmlu/default/kmmlu_mechanical_engineering.yaml' 2026-05-11T15:53:37,470 adding 'lm_eval/tasks/kmmlu/default/kmmlu_nondestructive_testing.yaml' 2026-05-11T15:53:37,471 adding 'lm_eval/tasks/kmmlu/default/kmmlu_patent.yaml' 2026-05-11T15:53:37,473 adding 'lm_eval/tasks/kmmlu/default/kmmlu_political_science_and_sociology.yaml' 2026-05-11T15:53:37,474 adding 'lm_eval/tasks/kmmlu/default/kmmlu_psychology.yaml' 2026-05-11T15:53:37,475 adding 'lm_eval/tasks/kmmlu/default/kmmlu_public_safety.yaml' 2026-05-11T15:53:37,476 adding 'lm_eval/tasks/kmmlu/default/kmmlu_railway_and_automotive_engineering.yaml' 2026-05-11T15:53:37,477 adding 'lm_eval/tasks/kmmlu/default/kmmlu_real_estate.yaml' 2026-05-11T15:53:37,478 adding 'lm_eval/tasks/kmmlu/default/kmmlu_refrigerating_machinery.yaml' 2026-05-11T15:53:37,480 adding 'lm_eval/tasks/kmmlu/default/kmmlu_social_welfare.yaml' 2026-05-11T15:53:37,481 adding 'lm_eval/tasks/kmmlu/default/kmmlu_taxation.yaml' 2026-05-11T15:53:37,482 adding 'lm_eval/tasks/kmmlu/default/kmmlu_telecommunications_and_wireless_technology.yaml' 2026-05-11T15:53:37,485 adding 'lm_eval/tasks/kmmlu/direct/_direct_kmmlu_yaml' 2026-05-11T15:53:37,486 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct.yaml' 2026-05-11T15:53:37,487 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_applied_science.yaml' 2026-05-11T15:53:37,488 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_humss.yaml' 2026-05-11T15:53:37,489 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_other.yaml' 2026-05-11T15:53:37,490 adding 'lm_eval/tasks/kmmlu/direct/_kmmlu_direct_stem.yaml' 2026-05-11T15:53:37,492 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_accounting.yaml' 2026-05-11T15:53:37,493 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_agricultural_sciences.yaml' 2026-05-11T15:53:37,494 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_aviation_engineering_and_maintenance.yaml' 2026-05-11T15:53:37,495 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_biology.yaml' 2026-05-11T15:53:37,496 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemical_engineering.yaml' 2026-05-11T15:53:37,498 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_chemistry.yaml' 2026-05-11T15:53:37,499 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_civil_engineering.yaml' 2026-05-11T15:53:37,500 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_computer_science.yaml' 2026-05-11T15:53:37,501 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_construction.yaml' 2026-05-11T15:53:37,502 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_criminal_law.yaml' 2026-05-11T15:53:37,503 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_ecology.yaml' 2026-05-11T15:53:37,505 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_economics.yaml' 2026-05-11T15:53:37,506 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_education.yaml' 2026-05-11T15:53:37,507 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_electrical_engineering.yaml' 2026-05-11T15:53:37,508 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_electronics_engineering.yaml' 2026-05-11T15:53:37,509 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_energy_management.yaml' 2026-05-11T15:53:37,510 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_environmental_science.yaml' 2026-05-11T15:53:37,512 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_fashion.yaml' 2026-05-11T15:53:37,513 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_food_processing.yaml' 2026-05-11T15:53:37,514 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_gas_technology_and_engineering.yaml' 2026-05-11T15:53:37,515 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_geomatics.yaml' 2026-05-11T15:53:37,516 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_health.yaml' 2026-05-11T15:53:37,518 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_industrial_engineer.yaml' 2026-05-11T15:53:37,519 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml' 2026-05-11T15:53:37,520 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml' 2026-05-11T15:53:37,521 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml' 2026-05-11T15:53:37,522 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml' 2026-05-11T15:53:37,523 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml' 2026-05-11T15:53:37,524 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml' 2026-05-11T15:53:37,525 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml' 2026-05-11T15:53:37,526 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml' 2026-05-11T15:53:37,528 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml' 2026-05-11T15:53:37,529 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml' 2026-05-11T15:53:37,530 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml' 2026-05-11T15:53:37,531 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml' 2026-05-11T15:53:37,532 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml' 2026-05-11T15:53:37,533 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml' 2026-05-11T15:53:37,535 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml' 2026-05-11T15:53:37,536 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml' 2026-05-11T15:53:37,537 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml' 2026-05-11T15:53:37,538 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml' 2026-05-11T15:53:37,539 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml' 2026-05-11T15:53:37,540 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml' 2026-05-11T15:53:37,541 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_taxation.yaml' 2026-05-11T15:53:37,543 adding 'lm_eval/tasks/kmmlu/direct/kmmlu_direct_telecommunications_and_wireless_technology.yaml' 2026-05-11T15:53:37,545 adding 'lm_eval/tasks/kmmlu/direct_hard/_direct_hard_kmmlu_yaml' 2026-05-11T15:53:37,546 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard.yaml' 2026-05-11T15:53:37,547 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_applied_science.yaml' 2026-05-11T15:53:37,549 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_humss.yaml' 2026-05-11T15:53:37,550 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_other.yaml' 2026-05-11T15:53:37,551 adding 'lm_eval/tasks/kmmlu/direct_hard/_kmmlu_direct_hard_stem.yaml' 2026-05-11T15:53:37,552 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_accounting.yaml' 2026-05-11T15:53:37,553 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_agricultural_sciences.yaml' 2026-05-11T15:53:37,555 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_aviation_engineering_and_maintenance.yaml' 2026-05-11T15:53:37,556 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_biology.yaml' 2026-05-11T15:53:37,557 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemical_engineering.yaml' 2026-05-11T15:53:37,558 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_chemistry.yaml' 2026-05-11T15:53:37,559 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_civil_engineering.yaml' 2026-05-11T15:53:37,561 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_computer_science.yaml' 2026-05-11T15:53:37,562 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_construction.yaml' 2026-05-11T15:53:37,563 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_criminal_law.yaml' 2026-05-11T15:53:37,564 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_ecology.yaml' 2026-05-11T15:53:37,566 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_economics.yaml' 2026-05-11T15:53:37,567 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_education.yaml' 2026-05-11T15:53:37,568 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electrical_engineering.yaml' 2026-05-11T15:53:37,569 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_electronics_engineering.yaml' 2026-05-11T15:53:37,570 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_energy_management.yaml' 2026-05-11T15:53:37,571 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_environmental_science.yaml' 2026-05-11T15:53:37,573 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_fashion.yaml' 2026-05-11T15:53:37,574 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_food_processing.yaml' 2026-05-11T15:53:37,575 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_gas_technology_and_engineering.yaml' 2026-05-11T15:53:37,576 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_geomatics.yaml' 2026-05-11T15:53:37,577 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_health.yaml' 2026-05-11T15:53:37,578 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_industrial_engineer.yaml' 2026-05-11T15:53:37,579 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_information_technology.yaml' 2026-05-11T15:53:37,581 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_interior_architecture_and_design.yaml' 2026-05-11T15:53:37,582 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_korean_history.yaml' 2026-05-11T15:53:37,583 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_law.yaml' 2026-05-11T15:53:37,584 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_machine_design_and_manufacturing.yaml' 2026-05-11T15:53:37,585 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_management.yaml' 2026-05-11T15:53:37,586 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_maritime_engineering.yaml' 2026-05-11T15:53:37,587 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_marketing.yaml' 2026-05-11T15:53:37,589 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_materials_engineering.yaml' 2026-05-11T15:53:37,590 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_math.yaml' 2026-05-11T15:53:37,591 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_mechanical_engineering.yaml' 2026-05-11T15:53:37,592 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_nondestructive_testing.yaml' 2026-05-11T15:53:37,594 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_patent.yaml' 2026-05-11T15:53:37,595 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_political_science_and_sociology.yaml' 2026-05-11T15:53:37,596 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_psychology.yaml' 2026-05-11T15:53:37,597 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_public_safety.yaml' 2026-05-11T15:53:37,599 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_railway_and_automotive_engineering.yaml' 2026-05-11T15:53:37,601 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_real_estate.yaml' 2026-05-11T15:53:37,602 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_refrigerating_machinery.yaml' 2026-05-11T15:53:37,603 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_social_welfare.yaml' 2026-05-11T15:53:37,605 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_taxation.yaml' 2026-05-11T15:53:37,606 adding 'lm_eval/tasks/kmmlu/direct_hard/kmmlu_direct_hard_telecommunications_and_wireless_technology.yaml' 2026-05-11T15:53:37,609 adding 'lm_eval/tasks/kmmlu/hard/_hard_kmmlu_yaml' 2026-05-11T15:53:37,610 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard.yaml' 2026-05-11T15:53:37,612 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_applied_science.yaml' 2026-05-11T15:53:37,613 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_humss.yaml' 2026-05-11T15:53:37,615 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_other.yaml' 2026-05-11T15:53:37,616 adding 'lm_eval/tasks/kmmlu/hard/_kmmlu_hard_stem.yaml' 2026-05-11T15:53:37,618 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_accounting.yaml' 2026-05-11T15:53:37,619 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_agricultural_sciences.yaml' 2026-05-11T15:53:37,621 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_aviation_engineering_and_maintenance.yaml' 2026-05-11T15:53:37,622 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_biology.yaml' 2026-05-11T15:53:37,624 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemical_engineering.yaml' 2026-05-11T15:53:37,625 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_chemistry.yaml' 2026-05-11T15:53:37,627 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_civil_engineering.yaml' 2026-05-11T15:53:37,628 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_computer_science.yaml' 2026-05-11T15:53:37,630 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_construction.yaml' 2026-05-11T15:53:37,631 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_criminal_law.yaml' 2026-05-11T15:53:37,633 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_ecology.yaml' 2026-05-11T15:53:37,634 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_economics.yaml' 2026-05-11T15:53:37,636 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_education.yaml' 2026-05-11T15:53:37,638 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_electrical_engineering.yaml' 2026-05-11T15:53:37,639 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_electronics_engineering.yaml' 2026-05-11T15:53:37,641 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_energy_management.yaml' 2026-05-11T15:53:37,643 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_environmental_science.yaml' 2026-05-11T15:53:37,644 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_fashion.yaml' 2026-05-11T15:53:37,646 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_food_processing.yaml' 2026-05-11T15:53:37,647 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_gas_technology_and_engineering.yaml' 2026-05-11T15:53:37,649 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_geomatics.yaml' 2026-05-11T15:53:37,650 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_health.yaml' 2026-05-11T15:53:37,652 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_industrial_engineer.yaml' 2026-05-11T15:53:37,654 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_information_technology.yaml' 2026-05-11T15:53:37,656 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_interior_architecture_and_design.yaml' 2026-05-11T15:53:37,657 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_korean_history.yaml' 2026-05-11T15:53:37,659 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_law.yaml' 2026-05-11T15:53:37,661 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_machine_design_and_manufacturing.yaml' 2026-05-11T15:53:37,662 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_management.yaml' 2026-05-11T15:53:37,664 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_maritime_engineering.yaml' 2026-05-11T15:53:37,665 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_marketing.yaml' 2026-05-11T15:53:37,667 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_materials_engineering.yaml' 2026-05-11T15:53:37,668 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_math.yaml' 2026-05-11T15:53:37,670 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_mechanical_engineering.yaml' 2026-05-11T15:53:37,672 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_nondestructive_testing.yaml' 2026-05-11T15:53:37,673 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_patent.yaml' 2026-05-11T15:53:37,675 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_political_science_and_sociology.yaml' 2026-05-11T15:53:37,676 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_psychology.yaml' 2026-05-11T15:53:37,678 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_public_safety.yaml' 2026-05-11T15:53:37,679 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_railway_and_automotive_engineering.yaml' 2026-05-11T15:53:37,681 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_real_estate.yaml' 2026-05-11T15:53:37,682 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_refrigerating_machinery.yaml' 2026-05-11T15:53:37,683 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_social_welfare.yaml' 2026-05-11T15:53:37,684 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_taxation.yaml' 2026-05-11T15:53:37,686 adding 'lm_eval/tasks/kmmlu/hard/kmmlu_hard_telecommunications_and_wireless_technology.yaml' 2026-05-11T15:53:37,688 adding 'lm_eval/tasks/kobest/README.md' 2026-05-11T15:53:37,689 adding 'lm_eval/tasks/kobest/_kobest.yaml' 2026-05-11T15:53:37,690 adding 'lm_eval/tasks/kobest/kobest_boolq.yaml' 2026-05-11T15:53:37,691 adding 'lm_eval/tasks/kobest/kobest_copa.yaml' 2026-05-11T15:53:37,693 adding 'lm_eval/tasks/kobest/kobest_hellaswag.yaml' 2026-05-11T15:53:37,694 adding 'lm_eval/tasks/kobest/kobest_sentineg.yaml' 2026-05-11T15:53:37,695 adding 'lm_eval/tasks/kobest/kobest_wic.yaml' 2026-05-11T15:53:37,696 adding 'lm_eval/tasks/kobest/utils.py' 2026-05-11T15:53:37,698 adding 'lm_eval/tasks/kormedmcqa/README.md' 2026-05-11T15:53:37,699 adding 'lm_eval/tasks/kormedmcqa/_kormedmcqa.yaml' 2026-05-11T15:53:37,701 adding 'lm_eval/tasks/kormedmcqa/_template_yaml' 2026-05-11T15:53:37,702 adding 'lm_eval/tasks/kormedmcqa/dentist.yaml' 2026-05-11T15:53:37,703 adding 'lm_eval/tasks/kormedmcqa/doctor.yaml' 2026-05-11T15:53:37,704 adding 'lm_eval/tasks/kormedmcqa/nurse.yaml' 2026-05-11T15:53:37,705 adding 'lm_eval/tasks/kormedmcqa/pharm.yaml' 2026-05-11T15:53:37,707 adding 'lm_eval/tasks/lambada/README.md' 2026-05-11T15:53:37,708 adding 'lm_eval/tasks/lambada/lambada_openai.yaml' 2026-05-11T15:53:37,710 adding 'lm_eval/tasks/lambada/lambada_standard.yaml' 2026-05-11T15:53:37,712 adding 'lm_eval/tasks/lambada_cloze/README.md' 2026-05-11T15:53:37,713 adding 'lm_eval/tasks/lambada_cloze/lambada_openai_cloze.yaml' 2026-05-11T15:53:37,714 adding 'lm_eval/tasks/lambada_cloze/lambada_standard_cloze.yaml' 2026-05-11T15:53:37,716 adding 'lm_eval/tasks/lambada_multilingual/README.md' 2026-05-11T15:53:37,717 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml' 2026-05-11T15:53:37,719 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml' 2026-05-11T15:53:37,720 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml' 2026-05-11T15:53:37,721 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml' 2026-05-11T15:53:37,722 adding 'lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml' 2026-05-11T15:53:37,724 adding 'lm_eval/tasks/lambada_multilingual_stablelm/README.md' 2026-05-11T15:53:37,725 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_de.yaml' 2026-05-11T15:53:37,726 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_en.yaml' 2026-05-11T15:53:37,728 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_es.yaml' 2026-05-11T15:53:37,729 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_fr.yaml' 2026-05-11T15:53:37,730 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_it.yaml' 2026-05-11T15:53:37,731 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_nl.yaml' 2026-05-11T15:53:37,732 adding 'lm_eval/tasks/lambada_multilingual_stablelm/lambada_mt_stablelm_pt.yaml' 2026-05-11T15:53:37,735 adding 'lm_eval/tasks/leaderboard/README.md' 2026-05-11T15:53:37,737 adding 'lm_eval/tasks/leaderboard/leaderboard.yaml' 2026-05-11T15:53:37,739 adding 'lm_eval/tasks/leaderboard/bbh_mc/_fewshot_template_yaml' 2026-05-11T15:53:37,740 adding 'lm_eval/tasks/leaderboard/bbh_mc/_leaderboard_bbh.yaml' 2026-05-11T15:53:37,741 adding 'lm_eval/tasks/leaderboard/bbh_mc/boolean_expressions.yaml' 2026-05-11T15:53:37,743 adding 'lm_eval/tasks/leaderboard/bbh_mc/causal_judgement.yaml' 2026-05-11T15:53:37,744 adding 'lm_eval/tasks/leaderboard/bbh_mc/date_understanding.yaml' 2026-05-11T15:53:37,745 adding 'lm_eval/tasks/leaderboard/bbh_mc/disambiguation_qa.yaml' 2026-05-11T15:53:37,747 adding 'lm_eval/tasks/leaderboard/bbh_mc/formal_fallacies.yaml' 2026-05-11T15:53:37,748 adding 'lm_eval/tasks/leaderboard/bbh_mc/geometric_shapes.yaml' 2026-05-11T15:53:37,750 adding 'lm_eval/tasks/leaderboard/bbh_mc/hyperbaton.yaml' 2026-05-11T15:53:37,751 adding 'lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_five_objects.yaml' 2026-05-11T15:53:37,752 adding 'lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_seven_objects.yaml' 2026-05-11T15:53:37,754 adding 'lm_eval/tasks/leaderboard/bbh_mc/logical_deduction_three_objects.yaml' 2026-05-11T15:53:37,755 adding 'lm_eval/tasks/leaderboard/bbh_mc/movie_recommendation.yaml' 2026-05-11T15:53:37,757 adding 'lm_eval/tasks/leaderboard/bbh_mc/navigate.yaml' 2026-05-11T15:53:37,758 adding 'lm_eval/tasks/leaderboard/bbh_mc/object_counting.yaml' 2026-05-11T15:53:37,759 adding 'lm_eval/tasks/leaderboard/bbh_mc/penguins_in_a_table.yaml' 2026-05-11T15:53:37,760 adding 'lm_eval/tasks/leaderboard/bbh_mc/reasoning_about_colored_objects.yaml' 2026-05-11T15:53:37,762 adding 'lm_eval/tasks/leaderboard/bbh_mc/ruin_names.yaml' 2026-05-11T15:53:37,763 adding 'lm_eval/tasks/leaderboard/bbh_mc/salient_translation_error_detection.yaml' 2026-05-11T15:53:37,764 adding 'lm_eval/tasks/leaderboard/bbh_mc/snarks.yaml' 2026-05-11T15:53:37,766 adding 'lm_eval/tasks/leaderboard/bbh_mc/sports_understanding.yaml' 2026-05-11T15:53:37,767 adding 'lm_eval/tasks/leaderboard/bbh_mc/temporal_sequences.yaml' 2026-05-11T15:53:37,768 adding 'lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_five_objects.yaml' 2026-05-11T15:53:37,770 adding 'lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_seven_objects.yaml' 2026-05-11T15:53:37,771 adding 'lm_eval/tasks/leaderboard/bbh_mc/tracking_shuffled_objects_three_objects.yaml' 2026-05-11T15:53:37,772 adding 'lm_eval/tasks/leaderboard/bbh_mc/web_of_lies.yaml' 2026-05-11T15:53:37,774 adding 'lm_eval/tasks/leaderboard/gpqa/_leaderboard_gpqa.yaml' 2026-05-11T15:53:37,775 adding 'lm_eval/tasks/leaderboard/gpqa/_template_yaml' 2026-05-11T15:53:37,777 adding 'lm_eval/tasks/leaderboard/gpqa/gpqa_diamond_zeroshot.yaml' 2026-05-11T15:53:37,778 adding 'lm_eval/tasks/leaderboard/gpqa/gpqa_extended_zeroshot.yaml' 2026-05-11T15:53:37,779 adding 'lm_eval/tasks/leaderboard/gpqa/gpqa_main_zeroshot.yaml' 2026-05-11T15:53:37,781 adding 'lm_eval/tasks/leaderboard/gpqa/utils.py' 2026-05-11T15:53:37,783 adding 'lm_eval/tasks/leaderboard/ifeval/_leaderboard_instruction_following.yaml' 2026-05-11T15:53:37,784 adding 'lm_eval/tasks/leaderboard/ifeval/ifeval.yaml' 2026-05-11T15:53:37,789 adding 'lm_eval/tasks/leaderboard/ifeval/instructions.py' 2026-05-11T15:53:37,791 adding 'lm_eval/tasks/leaderboard/ifeval/instructions_registry.py' 2026-05-11T15:53:37,795 adding 'lm_eval/tasks/leaderboard/ifeval/instructions_util.py' 2026-05-11T15:53:37,797 adding 'lm_eval/tasks/leaderboard/ifeval/utils.py' 2026-05-11T15:53:37,799 adding 'lm_eval/tasks/leaderboard/math/_leaderboard_math.yaml' 2026-05-11T15:53:37,800 adding 'lm_eval/tasks/leaderboard/math/_template_yaml' 2026-05-11T15:53:37,801 adding 'lm_eval/tasks/leaderboard/math/math_algebra.yaml' 2026-05-11T15:53:37,803 adding 'lm_eval/tasks/leaderboard/math/math_counting_and_prob.yaml' 2026-05-11T15:53:37,804 adding 'lm_eval/tasks/leaderboard/math/math_geometry.yaml' 2026-05-11T15:53:37,805 adding 'lm_eval/tasks/leaderboard/math/math_intermediate_algebra.yaml' 2026-05-11T15:53:37,806 adding 'lm_eval/tasks/leaderboard/math/math_num_theory.yaml' 2026-05-11T15:53:37,807 adding 'lm_eval/tasks/leaderboard/math/math_prealgebra.yaml' 2026-05-11T15:53:37,809 adding 'lm_eval/tasks/leaderboard/math/math_precalculus.yaml' 2026-05-11T15:53:37,811 adding 'lm_eval/tasks/leaderboard/math/utils.py' 2026-05-11T15:53:37,812 adding 'lm_eval/tasks/leaderboard/mmlu_pro/mmlu_pro.yaml' 2026-05-11T15:53:37,814 adding 'lm_eval/tasks/leaderboard/mmlu_pro/utils.py' 2026-05-11T15:53:37,815 adding 'lm_eval/tasks/leaderboard/musr/_musr.yaml' 2026-05-11T15:53:37,817 adding 'lm_eval/tasks/leaderboard/musr/_template_yaml' 2026-05-11T15:53:37,818 adding 'lm_eval/tasks/leaderboard/musr/musr_murder_mysteries.yaml' 2026-05-11T15:53:37,819 adding 'lm_eval/tasks/leaderboard/musr/musr_object_placements.yaml' 2026-05-11T15:53:37,820 adding 'lm_eval/tasks/leaderboard/musr/musr_team_allocation.yaml' 2026-05-11T15:53:37,822 adding 'lm_eval/tasks/leaderboard/musr/utils.py' 2026-05-11T15:53:37,824 adding 'lm_eval/tasks/libra/README.md' 2026-05-11T15:53:37,825 adding 'lm_eval/tasks/libra/_complex_reasoning_and_mathematical_problems.yaml' 2026-05-11T15:53:37,826 adding 'lm_eval/tasks/libra/_multi_hop_question_answering.yaml' 2026-05-11T15:53:37,828 adding 'lm_eval/tasks/libra/_question_answering_and_multiple_choice.yaml' 2026-05-11T15:53:37,829 adding 'lm_eval/tasks/libra/_simple_information_retrieval.yaml' 2026-05-11T15:53:37,830 adding 'lm_eval/tasks/libra/_template_yaml' 2026-05-11T15:53:37,831 adding 'lm_eval/tasks/libra/librusec_history.yaml' 2026-05-11T15:53:37,832 adding 'lm_eval/tasks/libra/librusec_mhqa.yaml' 2026-05-11T15:53:37,833 adding 'lm_eval/tasks/libra/long_context_multiq.yaml' 2026-05-11T15:53:37,835 adding 'lm_eval/tasks/libra/matreshka_names.yaml' 2026-05-11T15:53:37,836 adding 'lm_eval/tasks/libra/matreshka_yes_no.yaml' 2026-05-11T15:53:37,837 adding 'lm_eval/tasks/libra/passkey.yaml' 2026-05-11T15:53:37,838 adding 'lm_eval/tasks/libra/passkey_with_librusec.yaml' 2026-05-11T15:53:37,840 adding 'lm_eval/tasks/libra/ru_2wikimultihopqa.yaml' 2026-05-11T15:53:37,841 adding 'lm_eval/tasks/libra/ru_babilong_qa1.yaml' 2026-05-11T15:53:37,842 adding 'lm_eval/tasks/libra/ru_babilong_qa2.yaml' 2026-05-11T15:53:37,843 adding 'lm_eval/tasks/libra/ru_babilong_qa3.yaml' 2026-05-11T15:53:37,845 adding 'lm_eval/tasks/libra/ru_babilong_qa4.yaml' 2026-05-11T15:53:37,846 adding 'lm_eval/tasks/libra/ru_babilong_qa5.yaml' 2026-05-11T15:53:37,847 adding 'lm_eval/tasks/libra/ru_gsm100.yaml' 2026-05-11T15:53:37,848 adding 'lm_eval/tasks/libra/ru_qasper.yaml' 2026-05-11T15:53:37,850 adding 'lm_eval/tasks/libra/ru_quality.yaml' 2026-05-11T15:53:37,851 adding 'lm_eval/tasks/libra/ru_sci_abstract_retrieval.yaml' 2026-05-11T15:53:37,852 adding 'lm_eval/tasks/libra/ru_sci_passage_count.yaml' 2026-05-11T15:53:37,854 adding 'lm_eval/tasks/libra/utils.py' 2026-05-11T15:53:37,856 adding 'lm_eval/tasks/lingoly/README.md' 2026-05-11T15:53:37,857 adding 'lm_eval/tasks/lingoly/lingoly_context.yaml' 2026-05-11T15:53:37,858 adding 'lm_eval/tasks/lingoly/lingoly_group.yaml' 2026-05-11T15:53:37,860 adding 'lm_eval/tasks/lingoly/lingoly_nocontext.yaml' 2026-05-11T15:53:37,861 adding 'lm_eval/tasks/lingoly/script.py' 2026-05-11T15:53:37,863 adding 'lm_eval/tasks/lingoly/utils.py' 2026-05-11T15:53:37,865 adding 'lm_eval/tasks/llama3/README.md' 2026-05-11T15:53:37,867 adding 'lm_eval/tasks/llama3/instruct/arc_challenge/arc_challenge_llama.yaml' 2026-05-11T15:53:37,868 adding 'lm_eval/tasks/llama3/instruct/arc_challenge/utils.py' 2026-05-11T15:53:37,870 adding 'lm_eval/tasks/llama3/instruct/gsm8k/gsm8k.yaml' 2026-05-11T15:53:37,873 adding 'lm_eval/tasks/llama3/instruct/mmlu/_continuation_template_yaml' 2026-05-11T15:53:37,874 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_humanities.yaml' 2026-05-11T15:53:37,876 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_other.yaml' 2026-05-11T15:53:37,877 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_social_sciences.yaml' 2026-05-11T15:53:37,878 adding 'lm_eval/tasks/llama3/instruct/mmlu/_mmlu_stem.yaml' 2026-05-11T15:53:37,879 adding 'lm_eval/tasks/llama3/instruct/mmlu/llama.yaml' 2026-05-11T15:53:37,880 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:37,882 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_anatomy.yaml' 2026-05-11T15:53:37,883 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_astronomy.yaml' 2026-05-11T15:53:37,884 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_business_ethics.yaml' 2026-05-11T15:53:37,885 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:37,886 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_biology.yaml' 2026-05-11T15:53:37,888 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_chemistry.yaml' 2026-05-11T15:53:37,889 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_computer_science.yaml' 2026-05-11T15:53:37,890 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_mathematics.yaml' 2026-05-11T15:53:37,892 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_medicine.yaml' 2026-05-11T15:53:37,893 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_college_physics.yaml' 2026-05-11T15:53:37,894 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_computer_security.yaml' 2026-05-11T15:53:37,895 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:37,896 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_econometrics.yaml' 2026-05-11T15:53:37,897 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:37,898 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:37,900 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_formal_logic.yaml' 2026-05-11T15:53:37,901 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_global_facts.yaml' 2026-05-11T15:53:37,902 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_biology.yaml' 2026-05-11T15:53:37,903 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:37,904 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:37,905 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:37,907 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_geography.yaml' 2026-05-11T15:53:37,908 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:37,909 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:37,910 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:37,911 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:37,912 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_physics.yaml' 2026-05-11T15:53:37,914 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:37,915 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:37,916 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:37,917 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:37,918 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_aging.yaml' 2026-05-11T15:53:37,919 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_human_sexuality.yaml' 2026-05-11T15:53:37,921 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_international_law.yaml' 2026-05-11T15:53:37,922 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_jurisprudence.yaml' 2026-05-11T15:53:37,923 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:37,924 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_machine_learning.yaml' 2026-05-11T15:53:37,925 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_management.yaml' 2026-05-11T15:53:37,927 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_marketing.yaml' 2026-05-11T15:53:37,928 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_medical_genetics.yaml' 2026-05-11T15:53:37,929 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_miscellaneous.yaml' 2026-05-11T15:53:37,930 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_disputes.yaml' 2026-05-11T15:53:37,931 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:37,933 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_nutrition.yaml' 2026-05-11T15:53:37,934 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_philosophy.yaml' 2026-05-11T15:53:37,935 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_prehistory.yaml' 2026-05-11T15:53:37,936 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_accounting.yaml' 2026-05-11T15:53:37,938 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_law.yaml' 2026-05-11T15:53:37,939 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_medicine.yaml' 2026-05-11T15:53:37,940 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_professional_psychology.yaml' 2026-05-11T15:53:37,941 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_public_relations.yaml' 2026-05-11T15:53:37,943 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_security_studies.yaml' 2026-05-11T15:53:37,944 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_sociology.yaml' 2026-05-11T15:53:37,945 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:37,946 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_virology.yaml' 2026-05-11T15:53:37,947 adding 'lm_eval/tasks/llama3/instruct/mmlu/mmlu_world_religions.yaml' 2026-05-11T15:53:37,950 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama.yaml' 2026-05-11T15:53:37,951 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_cot_llama_template_yaml' 2026-05-11T15:53:37,952 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_humanities.yaml' 2026-05-11T15:53:37,953 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_other.yaml' 2026-05-11T15:53:37,954 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_social_sciences.yaml' 2026-05-11T15:53:37,956 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/_mmlu_stem.yaml' 2026-05-11T15:53:37,957 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:37,958 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_anatomy.yaml' 2026-05-11T15:53:37,959 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_astronomy.yaml' 2026-05-11T15:53:37,961 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_business_ethics.yaml' 2026-05-11T15:53:37,962 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:37,963 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_biology.yaml' 2026-05-11T15:53:37,964 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_chemistry.yaml' 2026-05-11T15:53:37,965 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_computer_science.yaml' 2026-05-11T15:53:37,966 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_mathematics.yaml' 2026-05-11T15:53:37,968 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_medicine.yaml' 2026-05-11T15:53:37,969 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_college_physics.yaml' 2026-05-11T15:53:37,970 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_computer_security.yaml' 2026-05-11T15:53:37,971 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:37,972 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_econometrics.yaml' 2026-05-11T15:53:37,974 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:37,975 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:37,976 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_formal_logic.yaml' 2026-05-11T15:53:37,977 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_global_facts.yaml' 2026-05-11T15:53:37,978 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_biology.yaml' 2026-05-11T15:53:37,980 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:37,981 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:37,982 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:37,983 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_geography.yaml' 2026-05-11T15:53:37,984 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:37,986 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:37,987 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:37,988 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:37,989 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_physics.yaml' 2026-05-11T15:53:37,990 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:37,991 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:37,993 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:37,994 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:37,995 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_aging.yaml' 2026-05-11T15:53:37,996 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_human_sexuality.yaml' 2026-05-11T15:53:37,997 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_international_law.yaml' 2026-05-11T15:53:37,998 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_jurisprudence.yaml' 2026-05-11T15:53:37,999 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:38,000 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_machine_learning.yaml' 2026-05-11T15:53:38,001 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_management.yaml' 2026-05-11T15:53:38,003 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_marketing.yaml' 2026-05-11T15:53:38,004 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_medical_genetics.yaml' 2026-05-11T15:53:38,005 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_miscellaneous.yaml' 2026-05-11T15:53:38,006 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_disputes.yaml' 2026-05-11T15:53:38,007 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:38,008 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_nutrition.yaml' 2026-05-11T15:53:38,009 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_philosophy.yaml' 2026-05-11T15:53:38,011 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_prehistory.yaml' 2026-05-11T15:53:38,012 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_accounting.yaml' 2026-05-11T15:53:38,013 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_law.yaml' 2026-05-11T15:53:38,014 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_medicine.yaml' 2026-05-11T15:53:38,015 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_professional_psychology.yaml' 2026-05-11T15:53:38,017 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_public_relations.yaml' 2026-05-11T15:53:38,018 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_security_studies.yaml' 2026-05-11T15:53:38,019 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_sociology.yaml' 2026-05-11T15:53:38,020 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:38,021 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_virology.yaml' 2026-05-11T15:53:38,023 adding 'lm_eval/tasks/llama3/instruct/mmlu_cot/mmlu_world_religions.yaml' 2026-05-11T15:53:38,026 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_continuation_template_yaml' 2026-05-11T15:53:38,027 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_humanities.yaml' 2026-05-11T15:53:38,028 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_llama.yaml' 2026-05-11T15:53:38,029 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_other.yaml' 2026-05-11T15:53:38,031 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_social_sciences.yaml' 2026-05-11T15:53:38,032 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/_mmlu_de_stem.yaml' 2026-05-11T15:53:38,033 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_abstract_algebra.yaml' 2026-05-11T15:53:38,035 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_anatomy.yaml' 2026-05-11T15:53:38,036 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_astronomy.yaml' 2026-05-11T15:53:38,037 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_business_ethics.yaml' 2026-05-11T15:53:38,039 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_clinical_knowledge.yaml' 2026-05-11T15:53:38,040 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_biology.yaml' 2026-05-11T15:53:38,042 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_chemistry.yaml' 2026-05-11T15:53:38,043 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_computer_science.yaml' 2026-05-11T15:53:38,044 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_mathematics.yaml' 2026-05-11T15:53:38,046 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_medicine.yaml' 2026-05-11T15:53:38,047 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_college_physics.yaml' 2026-05-11T15:53:38,048 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_computer_security.yaml' 2026-05-11T15:53:38,050 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_conceptual_physics.yaml' 2026-05-11T15:53:38,051 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_econometrics.yaml' 2026-05-11T15:53:38,052 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_electrical_engineering.yaml' 2026-05-11T15:53:38,054 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_elementary_mathematics.yaml' 2026-05-11T15:53:38,055 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_formal_logic.yaml' 2026-05-11T15:53:38,056 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_global_facts.yaml' 2026-05-11T15:53:38,058 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_biology.yaml' 2026-05-11T15:53:38,060 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_chemistry.yaml' 2026-05-11T15:53:38,061 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_computer_science.yaml' 2026-05-11T15:53:38,064 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_european_history.yaml' 2026-05-11T15:53:38,065 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_geography.yaml' 2026-05-11T15:53:38,067 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,068 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,070 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_mathematics.yaml' 2026-05-11T15:53:38,071 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_microeconomics.yaml' 2026-05-11T15:53:38,073 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_physics.yaml' 2026-05-11T15:53:38,074 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_psychology.yaml' 2026-05-11T15:53:38,076 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_statistics.yaml' 2026-05-11T15:53:38,079 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_us_history.yaml' 2026-05-11T15:53:38,081 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_high_school_world_history.yaml' 2026-05-11T15:53:38,082 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_aging.yaml' 2026-05-11T15:53:38,084 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_human_sexuality.yaml' 2026-05-11T15:53:38,085 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_international_law.yaml' 2026-05-11T15:53:38,086 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_jurisprudence.yaml' 2026-05-11T15:53:38,088 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_logical_fallacies.yaml' 2026-05-11T15:53:38,089 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_machine_learning.yaml' 2026-05-11T15:53:38,091 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_management.yaml' 2026-05-11T15:53:38,092 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_marketing.yaml' 2026-05-11T15:53:38,093 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_medical_genetics.yaml' 2026-05-11T15:53:38,095 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_miscellaneous.yaml' 2026-05-11T15:53:38,096 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_disputes.yaml' 2026-05-11T15:53:38,097 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_moral_scenarios.yaml' 2026-05-11T15:53:38,099 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_nutrition.yaml' 2026-05-11T15:53:38,100 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_philosophy.yaml' 2026-05-11T15:53:38,101 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_prehistory.yaml' 2026-05-11T15:53:38,103 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_accounting.yaml' 2026-05-11T15:53:38,105 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_law.yaml' 2026-05-11T15:53:38,106 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_medicine.yaml' 2026-05-11T15:53:38,108 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_professional_psychology.yaml' 2026-05-11T15:53:38,109 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_public_relations.yaml' 2026-05-11T15:53:38,111 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_security_studies.yaml' 2026-05-11T15:53:38,112 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_sociology.yaml' 2026-05-11T15:53:38,114 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_us_foreign_policy.yaml' 2026-05-11T15:53:38,115 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_virology.yaml' 2026-05-11T15:53:38,117 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/mmlu_de_world_religions.yaml' 2026-05-11T15:53:38,118 adding 'lm_eval/tasks/llama3/instruct/mmlu_de/utils.py' 2026-05-11T15:53:38,121 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_continuation_template_yaml' 2026-05-11T15:53:38,122 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_humanities.yaml' 2026-05-11T15:53:38,123 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_llama.yaml' 2026-05-11T15:53:38,124 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_other.yaml' 2026-05-11T15:53:38,125 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_social_sciences.yaml' 2026-05-11T15:53:38,127 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/_mmlu_es_stem.yaml' 2026-05-11T15:53:38,128 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_abstract_algebra.yaml' 2026-05-11T15:53:38,129 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_anatomy.yaml' 2026-05-11T15:53:38,131 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_astronomy.yaml' 2026-05-11T15:53:38,132 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_business_ethics.yaml' 2026-05-11T15:53:38,133 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_clinical_knowledge.yaml' 2026-05-11T15:53:38,135 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_biology.yaml' 2026-05-11T15:53:38,136 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_chemistry.yaml' 2026-05-11T15:53:38,138 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_computer_science.yaml' 2026-05-11T15:53:38,139 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_mathematics.yaml' 2026-05-11T15:53:38,141 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_medicine.yaml' 2026-05-11T15:53:38,142 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_college_physics.yaml' 2026-05-11T15:53:38,144 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_computer_security.yaml' 2026-05-11T15:53:38,145 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_conceptual_physics.yaml' 2026-05-11T15:53:38,146 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_econometrics.yaml' 2026-05-11T15:53:38,148 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_electrical_engineering.yaml' 2026-05-11T15:53:38,149 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_elementary_mathematics.yaml' 2026-05-11T15:53:38,150 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_formal_logic.yaml' 2026-05-11T15:53:38,152 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_global_facts.yaml' 2026-05-11T15:53:38,153 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_biology.yaml' 2026-05-11T15:53:38,154 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_chemistry.yaml' 2026-05-11T15:53:38,156 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_computer_science.yaml' 2026-05-11T15:53:38,159 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_european_history.yaml' 2026-05-11T15:53:38,160 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_geography.yaml' 2026-05-11T15:53:38,162 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,163 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,164 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_mathematics.yaml' 2026-05-11T15:53:38,166 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_microeconomics.yaml' 2026-05-11T15:53:38,167 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_physics.yaml' 2026-05-11T15:53:38,168 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_psychology.yaml' 2026-05-11T15:53:38,170 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_statistics.yaml' 2026-05-11T15:53:38,172 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml' 2026-05-11T15:53:38,174 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml' 2026-05-11T15:53:38,176 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml' 2026-05-11T15:53:38,177 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml' 2026-05-11T15:53:38,178 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml' 2026-05-11T15:53:38,179 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml' 2026-05-11T15:53:38,181 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml' 2026-05-11T15:53:38,182 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml' 2026-05-11T15:53:38,183 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml' 2026-05-11T15:53:38,185 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml' 2026-05-11T15:53:38,186 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml' 2026-05-11T15:53:38,187 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml' 2026-05-11T15:53:38,189 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml' 2026-05-11T15:53:38,190 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml' 2026-05-11T15:53:38,191 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml' 2026-05-11T15:53:38,193 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml' 2026-05-11T15:53:38,194 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml' 2026-05-11T15:53:38,196 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml' 2026-05-11T15:53:38,198 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml' 2026-05-11T15:53:38,199 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml' 2026-05-11T15:53:38,201 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_psychology.yaml' 2026-05-11T15:53:38,202 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_public_relations.yaml' 2026-05-11T15:53:38,204 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_security_studies.yaml' 2026-05-11T15:53:38,205 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_sociology.yaml' 2026-05-11T15:53:38,207 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_us_foreign_policy.yaml' 2026-05-11T15:53:38,208 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_virology.yaml' 2026-05-11T15:53:38,209 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_world_religions.yaml' 2026-05-11T15:53:38,211 adding 'lm_eval/tasks/llama3/instruct/mmlu_es/utils.py' 2026-05-11T15:53:38,214 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_continuation_template_yaml' 2026-05-11T15:53:38,215 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_humanities.yaml' 2026-05-11T15:53:38,217 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_llama.yaml' 2026-05-11T15:53:38,218 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_other.yaml' 2026-05-11T15:53:38,219 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_social_sciences.yaml' 2026-05-11T15:53:38,220 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/_mmlu_fr_stem.yaml' 2026-05-11T15:53:38,221 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_abstract_algebra.yaml' 2026-05-11T15:53:38,223 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_anatomy.yaml' 2026-05-11T15:53:38,224 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_astronomy.yaml' 2026-05-11T15:53:38,225 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_business_ethics.yaml' 2026-05-11T15:53:38,227 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_clinical_knowledge.yaml' 2026-05-11T15:53:38,228 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_biology.yaml' 2026-05-11T15:53:38,230 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml' 2026-05-11T15:53:38,231 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml' 2026-05-11T15:53:38,233 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml' 2026-05-11T15:53:38,234 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml' 2026-05-11T15:53:38,235 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml' 2026-05-11T15:53:38,237 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml' 2026-05-11T15:53:38,238 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml' 2026-05-11T15:53:38,239 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml' 2026-05-11T15:53:38,241 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml' 2026-05-11T15:53:38,242 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml' 2026-05-11T15:53:38,243 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml' 2026-05-11T15:53:38,245 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml' 2026-05-11T15:53:38,246 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml' 2026-05-11T15:53:38,247 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml' 2026-05-11T15:53:38,249 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml' 2026-05-11T15:53:38,252 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml' 2026-05-11T15:53:38,253 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml' 2026-05-11T15:53:38,255 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,256 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,258 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml' 2026-05-11T15:53:38,259 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_microeconomics.yaml' 2026-05-11T15:53:38,260 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_physics.yaml' 2026-05-11T15:53:38,262 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_psychology.yaml' 2026-05-11T15:53:38,263 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_statistics.yaml' 2026-05-11T15:53:38,266 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_us_history.yaml' 2026-05-11T15:53:38,268 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_world_history.yaml' 2026-05-11T15:53:38,269 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_aging.yaml' 2026-05-11T15:53:38,271 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_human_sexuality.yaml' 2026-05-11T15:53:38,272 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_international_law.yaml' 2026-05-11T15:53:38,274 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_jurisprudence.yaml' 2026-05-11T15:53:38,275 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_logical_fallacies.yaml' 2026-05-11T15:53:38,276 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_machine_learning.yaml' 2026-05-11T15:53:38,277 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_management.yaml' 2026-05-11T15:53:38,279 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_marketing.yaml' 2026-05-11T15:53:38,280 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_medical_genetics.yaml' 2026-05-11T15:53:38,281 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_miscellaneous.yaml' 2026-05-11T15:53:38,283 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_disputes.yaml' 2026-05-11T15:53:38,284 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_moral_scenarios.yaml' 2026-05-11T15:53:38,285 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_nutrition.yaml' 2026-05-11T15:53:38,287 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_philosophy.yaml' 2026-05-11T15:53:38,288 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_prehistory.yaml' 2026-05-11T15:53:38,290 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_accounting.yaml' 2026-05-11T15:53:38,292 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_law.yaml' 2026-05-11T15:53:38,293 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_medicine.yaml' 2026-05-11T15:53:38,295 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_professional_psychology.yaml' 2026-05-11T15:53:38,296 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_public_relations.yaml' 2026-05-11T15:53:38,298 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_security_studies.yaml' 2026-05-11T15:53:38,299 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_sociology.yaml' 2026-05-11T15:53:38,301 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_us_foreign_policy.yaml' 2026-05-11T15:53:38,302 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_virology.yaml' 2026-05-11T15:53:38,303 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_world_religions.yaml' 2026-05-11T15:53:38,305 adding 'lm_eval/tasks/llama3/instruct/mmlu_fr/utils.py' 2026-05-11T15:53:38,308 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_continuation_template_yaml' 2026-05-11T15:53:38,309 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_humanities.yaml' 2026-05-11T15:53:38,310 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_llama.yaml' 2026-05-11T15:53:38,311 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_other.yaml' 2026-05-11T15:53:38,312 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_social_sciences.yaml' 2026-05-11T15:53:38,314 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/_mmlu_hi_stem.yaml' 2026-05-11T15:53:38,315 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_abstract_algebra.yaml' 2026-05-11T15:53:38,317 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_anatomy.yaml' 2026-05-11T15:53:38,319 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_astronomy.yaml' 2026-05-11T15:53:38,321 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_business_ethics.yaml' 2026-05-11T15:53:38,323 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_clinical_knowledge.yaml' 2026-05-11T15:53:38,325 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_biology.yaml' 2026-05-11T15:53:38,327 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_chemistry.yaml' 2026-05-11T15:53:38,329 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_computer_science.yaml' 2026-05-11T15:53:38,331 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_mathematics.yaml' 2026-05-11T15:53:38,333 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_medicine.yaml' 2026-05-11T15:53:38,335 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_college_physics.yaml' 2026-05-11T15:53:38,337 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_computer_security.yaml' 2026-05-11T15:53:38,339 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_conceptual_physics.yaml' 2026-05-11T15:53:38,341 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_econometrics.yaml' 2026-05-11T15:53:38,342 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_electrical_engineering.yaml' 2026-05-11T15:53:38,344 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_elementary_mathematics.yaml' 2026-05-11T15:53:38,346 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_formal_logic.yaml' 2026-05-11T15:53:38,348 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_global_facts.yaml' 2026-05-11T15:53:38,350 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_biology.yaml' 2026-05-11T15:53:38,351 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_chemistry.yaml' 2026-05-11T15:53:38,354 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_computer_science.yaml' 2026-05-11T15:53:38,363 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_european_history.yaml' 2026-05-11T15:53:38,365 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_geography.yaml' 2026-05-11T15:53:38,368 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,369 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,371 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_mathematics.yaml' 2026-05-11T15:53:38,373 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_microeconomics.yaml' 2026-05-11T15:53:38,374 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_physics.yaml' 2026-05-11T15:53:38,376 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_psychology.yaml' 2026-05-11T15:53:38,379 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_statistics.yaml' 2026-05-11T15:53:38,386 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_us_history.yaml' 2026-05-11T15:53:38,391 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_high_school_world_history.yaml' 2026-05-11T15:53:38,392 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_aging.yaml' 2026-05-11T15:53:38,394 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_human_sexuality.yaml' 2026-05-11T15:53:38,396 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_international_law.yaml' 2026-05-11T15:53:38,398 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_jurisprudence.yaml' 2026-05-11T15:53:38,400 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_logical_fallacies.yaml' 2026-05-11T15:53:38,403 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_machine_learning.yaml' 2026-05-11T15:53:38,404 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_management.yaml' 2026-05-11T15:53:38,406 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_marketing.yaml' 2026-05-11T15:53:38,408 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_medical_genetics.yaml' 2026-05-11T15:53:38,410 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_miscellaneous.yaml' 2026-05-11T15:53:38,412 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_disputes.yaml' 2026-05-11T15:53:38,413 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_moral_scenarios.yaml' 2026-05-11T15:53:38,416 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_nutrition.yaml' 2026-05-11T15:53:38,417 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_philosophy.yaml' 2026-05-11T15:53:38,419 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_prehistory.yaml' 2026-05-11T15:53:38,422 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_accounting.yaml' 2026-05-11T15:53:38,427 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_law.yaml' 2026-05-11T15:53:38,431 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_medicine.yaml' 2026-05-11T15:53:38,433 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_professional_psychology.yaml' 2026-05-11T15:53:38,435 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_public_relations.yaml' 2026-05-11T15:53:38,439 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_security_studies.yaml' 2026-05-11T15:53:38,442 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_sociology.yaml' 2026-05-11T15:53:38,444 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_us_foreign_policy.yaml' 2026-05-11T15:53:38,445 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_virology.yaml' 2026-05-11T15:53:38,447 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/mmlu_hi_world_religions.yaml' 2026-05-11T15:53:38,448 adding 'lm_eval/tasks/llama3/instruct/mmlu_hi/utils.py' 2026-05-11T15:53:38,451 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_continuation_template_yaml' 2026-05-11T15:53:38,452 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_humanities.yaml' 2026-05-11T15:53:38,453 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_llama.yaml' 2026-05-11T15:53:38,454 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_other.yaml' 2026-05-11T15:53:38,456 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_social_sciences.yaml' 2026-05-11T15:53:38,457 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/_mmlu_it_stem.yaml' 2026-05-11T15:53:38,458 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_abstract_algebra.yaml' 2026-05-11T15:53:38,459 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_anatomy.yaml' 2026-05-11T15:53:38,461 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_astronomy.yaml' 2026-05-11T15:53:38,462 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_business_ethics.yaml' 2026-05-11T15:53:38,463 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_clinical_knowledge.yaml' 2026-05-11T15:53:38,465 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_biology.yaml' 2026-05-11T15:53:38,466 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_chemistry.yaml' 2026-05-11T15:53:38,468 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_computer_science.yaml' 2026-05-11T15:53:38,469 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_mathematics.yaml' 2026-05-11T15:53:38,470 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_medicine.yaml' 2026-05-11T15:53:38,472 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_college_physics.yaml' 2026-05-11T15:53:38,473 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_computer_security.yaml' 2026-05-11T15:53:38,475 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_conceptual_physics.yaml' 2026-05-11T15:53:38,476 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_econometrics.yaml' 2026-05-11T15:53:38,477 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_electrical_engineering.yaml' 2026-05-11T15:53:38,479 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_elementary_mathematics.yaml' 2026-05-11T15:53:38,480 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_formal_logic.yaml' 2026-05-11T15:53:38,482 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_global_facts.yaml' 2026-05-11T15:53:38,483 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_biology.yaml' 2026-05-11T15:53:38,484 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_chemistry.yaml' 2026-05-11T15:53:38,486 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_computer_science.yaml' 2026-05-11T15:53:38,488 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_european_history.yaml' 2026-05-11T15:53:38,490 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_geography.yaml' 2026-05-11T15:53:38,491 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,492 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,494 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_mathematics.yaml' 2026-05-11T15:53:38,495 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_microeconomics.yaml' 2026-05-11T15:53:38,497 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_physics.yaml' 2026-05-11T15:53:38,498 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_psychology.yaml' 2026-05-11T15:53:38,499 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_statistics.yaml' 2026-05-11T15:53:38,502 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_us_history.yaml' 2026-05-11T15:53:38,503 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_high_school_world_history.yaml' 2026-05-11T15:53:38,505 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_aging.yaml' 2026-05-11T15:53:38,506 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_human_sexuality.yaml' 2026-05-11T15:53:38,507 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_international_law.yaml' 2026-05-11T15:53:38,509 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_jurisprudence.yaml' 2026-05-11T15:53:38,510 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_logical_fallacies.yaml' 2026-05-11T15:53:38,511 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_machine_learning.yaml' 2026-05-11T15:53:38,513 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_management.yaml' 2026-05-11T15:53:38,514 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_marketing.yaml' 2026-05-11T15:53:38,515 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_medical_genetics.yaml' 2026-05-11T15:53:38,517 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_miscellaneous.yaml' 2026-05-11T15:53:38,518 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_disputes.yaml' 2026-05-11T15:53:38,519 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_moral_scenarios.yaml' 2026-05-11T15:53:38,521 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_nutrition.yaml' 2026-05-11T15:53:38,522 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_philosophy.yaml' 2026-05-11T15:53:38,523 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_prehistory.yaml' 2026-05-11T15:53:38,525 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_accounting.yaml' 2026-05-11T15:53:38,527 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_law.yaml' 2026-05-11T15:53:38,528 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_medicine.yaml' 2026-05-11T15:53:38,530 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_professional_psychology.yaml' 2026-05-11T15:53:38,531 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_public_relations.yaml' 2026-05-11T15:53:38,533 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_security_studies.yaml' 2026-05-11T15:53:38,534 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_sociology.yaml' 2026-05-11T15:53:38,536 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_us_foreign_policy.yaml' 2026-05-11T15:53:38,537 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_virology.yaml' 2026-05-11T15:53:38,538 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/mmlu_it_world_religions.yaml' 2026-05-11T15:53:38,540 adding 'lm_eval/tasks/llama3/instruct/mmlu_it/utils.py' 2026-05-11T15:53:38,542 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/_default_template_yaml' 2026-05-11T15:53:38,543 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/_mmlu_pro.yaml' 2026-05-11T15:53:38,544 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_biology.yaml' 2026-05-11T15:53:38,545 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_business.yaml' 2026-05-11T15:53:38,547 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_chemistry.yaml' 2026-05-11T15:53:38,548 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_computer_science.yaml' 2026-05-11T15:53:38,549 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_economics.yaml' 2026-05-11T15:53:38,550 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_engineering.yaml' 2026-05-11T15:53:38,551 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_health.yaml' 2026-05-11T15:53:38,553 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_history.yaml' 2026-05-11T15:53:38,554 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_law.yaml' 2026-05-11T15:53:38,555 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_math.yaml' 2026-05-11T15:53:38,556 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_other.yaml' 2026-05-11T15:53:38,557 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_philosophy.yaml' 2026-05-11T15:53:38,558 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_physics.yaml' 2026-05-11T15:53:38,559 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/mmlu_pro_psychology.yaml' 2026-05-11T15:53:38,560 adding 'lm_eval/tasks/llama3/instruct/mmlu_pro/utils.py' 2026-05-11T15:53:38,563 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_continuation_template_yaml' 2026-05-11T15:53:38,564 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_humanities.yaml' 2026-05-11T15:53:38,565 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_llama.yaml' 2026-05-11T15:53:38,567 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_other.yaml' 2026-05-11T15:53:38,568 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_social_sciences.yaml' 2026-05-11T15:53:38,569 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/_mmlu_pt_stem.yaml' 2026-05-11T15:53:38,570 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_abstract_algebra.yaml' 2026-05-11T15:53:38,572 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_anatomy.yaml' 2026-05-11T15:53:38,573 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_astronomy.yaml' 2026-05-11T15:53:38,574 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_business_ethics.yaml' 2026-05-11T15:53:38,576 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_clinical_knowledge.yaml' 2026-05-11T15:53:38,577 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_biology.yaml' 2026-05-11T15:53:38,579 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_chemistry.yaml' 2026-05-11T15:53:38,580 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_computer_science.yaml' 2026-05-11T15:53:38,582 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_mathematics.yaml' 2026-05-11T15:53:38,583 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_medicine.yaml' 2026-05-11T15:53:38,584 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_college_physics.yaml' 2026-05-11T15:53:38,586 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_computer_security.yaml' 2026-05-11T15:53:38,587 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_conceptual_physics.yaml' 2026-05-11T15:53:38,588 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_econometrics.yaml' 2026-05-11T15:53:38,590 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_electrical_engineering.yaml' 2026-05-11T15:53:38,591 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_elementary_mathematics.yaml' 2026-05-11T15:53:38,593 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_formal_logic.yaml' 2026-05-11T15:53:38,594 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_global_facts.yaml' 2026-05-11T15:53:38,595 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_biology.yaml' 2026-05-11T15:53:38,597 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_chemistry.yaml' 2026-05-11T15:53:38,599 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_computer_science.yaml' 2026-05-11T15:53:38,601 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_european_history.yaml' 2026-05-11T15:53:38,603 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_geography.yaml' 2026-05-11T15:53:38,604 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,605 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,607 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_mathematics.yaml' 2026-05-11T15:53:38,608 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_microeconomics.yaml' 2026-05-11T15:53:38,610 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_physics.yaml' 2026-05-11T15:53:38,611 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_psychology.yaml' 2026-05-11T15:53:38,612 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_statistics.yaml' 2026-05-11T15:53:38,615 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_us_history.yaml' 2026-05-11T15:53:38,616 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_high_school_world_history.yaml' 2026-05-11T15:53:38,618 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_aging.yaml' 2026-05-11T15:53:38,619 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_human_sexuality.yaml' 2026-05-11T15:53:38,620 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_international_law.yaml' 2026-05-11T15:53:38,622 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_jurisprudence.yaml' 2026-05-11T15:53:38,623 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_logical_fallacies.yaml' 2026-05-11T15:53:38,625 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_machine_learning.yaml' 2026-05-11T15:53:38,626 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_management.yaml' 2026-05-11T15:53:38,627 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_marketing.yaml' 2026-05-11T15:53:38,629 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_medical_genetics.yaml' 2026-05-11T15:53:38,630 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_miscellaneous.yaml' 2026-05-11T15:53:38,631 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_disputes.yaml' 2026-05-11T15:53:38,632 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_moral_scenarios.yaml' 2026-05-11T15:53:38,634 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_nutrition.yaml' 2026-05-11T15:53:38,635 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_philosophy.yaml' 2026-05-11T15:53:38,636 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_prehistory.yaml' 2026-05-11T15:53:38,638 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_accounting.yaml' 2026-05-11T15:53:38,640 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_law.yaml' 2026-05-11T15:53:38,642 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_medicine.yaml' 2026-05-11T15:53:38,643 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_professional_psychology.yaml' 2026-05-11T15:53:38,644 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_public_relations.yaml' 2026-05-11T15:53:38,646 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_security_studies.yaml' 2026-05-11T15:53:38,648 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_sociology.yaml' 2026-05-11T15:53:38,649 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_us_foreign_policy.yaml' 2026-05-11T15:53:38,651 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_virology.yaml' 2026-05-11T15:53:38,652 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/mmlu_pt_world_religions.yaml' 2026-05-11T15:53:38,653 adding 'lm_eval/tasks/llama3/instruct/mmlu_pt/utils.py' 2026-05-11T15:53:38,656 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_continuation_template_yaml' 2026-05-11T15:53:38,658 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_humanities.yaml' 2026-05-11T15:53:38,659 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_llama.yaml' 2026-05-11T15:53:38,660 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_other.yaml' 2026-05-11T15:53:38,661 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_social_sciences.yaml' 2026-05-11T15:53:38,663 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/_mmlu_th_stem.yaml' 2026-05-11T15:53:38,664 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_abstract_algebra.yaml' 2026-05-11T15:53:38,666 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_anatomy.yaml' 2026-05-11T15:53:38,668 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_astronomy.yaml' 2026-05-11T15:53:38,670 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_business_ethics.yaml' 2026-05-11T15:53:38,672 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_clinical_knowledge.yaml' 2026-05-11T15:53:38,674 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_biology.yaml' 2026-05-11T15:53:38,675 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_chemistry.yaml' 2026-05-11T15:53:38,678 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_computer_science.yaml' 2026-05-11T15:53:38,679 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_mathematics.yaml' 2026-05-11T15:53:38,682 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_medicine.yaml' 2026-05-11T15:53:38,683 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_college_physics.yaml' 2026-05-11T15:53:38,685 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_computer_security.yaml' 2026-05-11T15:53:38,686 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_conceptual_physics.yaml' 2026-05-11T15:53:38,688 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_econometrics.yaml' 2026-05-11T15:53:38,690 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_electrical_engineering.yaml' 2026-05-11T15:53:38,692 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_elementary_mathematics.yaml' 2026-05-11T15:53:38,694 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_formal_logic.yaml' 2026-05-11T15:53:38,695 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_global_facts.yaml' 2026-05-11T15:53:38,697 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_biology.yaml' 2026-05-11T15:53:38,699 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_chemistry.yaml' 2026-05-11T15:53:38,701 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_computer_science.yaml' 2026-05-11T15:53:38,711 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_european_history.yaml' 2026-05-11T15:53:38,713 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_geography.yaml' 2026-05-11T15:53:38,715 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_government_and_politics.yaml' 2026-05-11T15:53:38,717 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_macroeconomics.yaml' 2026-05-11T15:53:38,719 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_mathematics.yaml' 2026-05-11T15:53:38,721 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_microeconomics.yaml' 2026-05-11T15:53:38,722 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_physics.yaml' 2026-05-11T15:53:38,724 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_psychology.yaml' 2026-05-11T15:53:38,727 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_statistics.yaml' 2026-05-11T15:53:38,734 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_us_history.yaml' 2026-05-11T15:53:38,739 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_high_school_world_history.yaml' 2026-05-11T15:53:38,741 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_aging.yaml' 2026-05-11T15:53:38,742 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_human_sexuality.yaml' 2026-05-11T15:53:38,745 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_international_law.yaml' 2026-05-11T15:53:38,746 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_jurisprudence.yaml' 2026-05-11T15:53:38,749 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_logical_fallacies.yaml' 2026-05-11T15:53:38,751 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_machine_learning.yaml' 2026-05-11T15:53:38,752 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_management.yaml' 2026-05-11T15:53:38,754 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_marketing.yaml' 2026-05-11T15:53:38,756 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_medical_genetics.yaml' 2026-05-11T15:53:38,758 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_miscellaneous.yaml' 2026-05-11T15:53:38,760 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_disputes.yaml' 2026-05-11T15:53:38,762 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_moral_scenarios.yaml' 2026-05-11T15:53:38,764 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_nutrition.yaml' 2026-05-11T15:53:38,766 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_philosophy.yaml' 2026-05-11T15:53:38,768 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_prehistory.yaml' 2026-05-11T15:53:38,770 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_accounting.yaml' 2026-05-11T15:53:38,775 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_law.yaml' 2026-05-11T15:53:38,779 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_medicine.yaml' 2026-05-11T15:53:38,781 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml' 2026-05-11T15:53:38,783 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml' 2026-05-11T15:53:38,788 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml' 2026-05-11T15:53:38,790 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml' 2026-05-11T15:53:38,792 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml' 2026-05-11T15:53:38,793 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml' 2026-05-11T15:53:38,795 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml' 2026-05-11T15:53:38,796 adding 'lm_eval/tasks/llama3/instruct/mmlu_th/utils.py' 2026-05-11T15:53:38,801 adding 'lm_eval/tasks/lm_syneval/README.md' 2026-05-11T15:53:38,802 adding 'lm_eval/tasks/lm_syneval/_template_yaml' 2026-05-11T15:53:38,803 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__plur_MS_LMV_LMV.yaml' 2026-05-11T15:53:38,804 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__long_vp_coord__sing_MS_LMV_LMV.yaml' 2026-05-11T15:53:38,806 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_plur_ES_EV.yaml' 2026-05-11T15:53:38,807 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__plur_MS_MV_sing_ES_EV.yaml' 2026-05-11T15:53:38,808 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_plur_ES_EV.yaml' 2026-05-11T15:53:38,809 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_anim__sing_MS_MV_sing_ES_EV.yaml' 2026-05-11T15:53:38,810 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_plur_ES_EV.yaml' 2026-05-11T15:53:38,812 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__plur_IS_IV_sing_ES_EV.yaml' 2026-05-11T15:53:38,813 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_plur_ES_EV.yaml' 2026-05-11T15:53:38,814 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_across_inanim__sing_IS_IV_sing_ES_EV.yaml' 2026-05-11T15:53:38,815 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_plur_ES_EV.yaml' 2026-05-11T15:53:38,816 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__plur_MS_MV_sing_ES_EV.yaml' 2026-05-11T15:53:38,818 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_plur_ES_EV.yaml' 2026-05-11T15:53:38,819 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_anim__sing_MS_MV_sing_ES_EV.yaml' 2026-05-11T15:53:38,820 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_plur_ES_EV.yaml' 2026-05-11T15:53:38,822 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__plur_IS_IV_sing_ES_EV.yaml' 2026-05-11T15:53:38,823 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_plur_ES_EV.yaml' 2026-05-11T15:53:38,824 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_across_inanim__sing_IS_IV_sing_ES_EV.yaml' 2026-05-11T15:53:38,825 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_plur_MS_MV.yaml' 2026-05-11T15:53:38,826 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__plur_ES_EV_sing_MS_MV.yaml' 2026-05-11T15:53:38,827 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_plur_MS_MV.yaml' 2026-05-11T15:53:38,829 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_anim__sing_ES_EV_sing_MS_MV.yaml' 2026-05-11T15:53:38,830 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_plur_IS_IV.yaml' 2026-05-11T15:53:38,831 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__plur_ES_EV_sing_IS_IV.yaml' 2026-05-11T15:53:38,832 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_plur_IS_IV.yaml' 2026-05-11T15:53:38,833 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_no_comp_within_inanim__sing_ES_EV_sing_IS_IV.yaml' 2026-05-11T15:53:38,834 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_plur_MS_MV.yaml' 2026-05-11T15:53:38,835 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__plur_ES_EV_sing_MS_MV.yaml' 2026-05-11T15:53:38,836 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_plur_MS_MV.yaml' 2026-05-11T15:53:38,838 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_anim__sing_ES_EV_sing_MS_MV.yaml' 2026-05-11T15:53:38,839 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_plur_IS_IV.yaml' 2026-05-11T15:53:38,840 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__plur_ES_EV_sing_IS_IV.yaml' 2026-05-11T15:53:38,841 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_plur_IS_IV.yaml' 2026-05-11T15:53:38,842 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__obj_rel_within_inanim__sing_ES_EV_sing_IS_IV.yaml' 2026-05-11T15:53:38,843 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_plur_ES.yaml' 2026-05-11T15:53:38,845 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__plur_MS_MV_sing_ES.yaml' 2026-05-11T15:53:38,846 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_plur_ES.yaml' 2026-05-11T15:53:38,847 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_anim__sing_MS_MV_sing_ES.yaml' 2026-05-11T15:53:38,848 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_plur_ES.yaml' 2026-05-11T15:53:38,850 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__plur_IS_IV_sing_ES.yaml' 2026-05-11T15:53:38,851 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_plur_ES.yaml' 2026-05-11T15:53:38,852 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__prep_inanim__sing_IS_IV_sing_ES.yaml' 2026-05-11T15:53:38,853 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_plur_BS.yaml' 2026-05-11T15:53:38,854 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__plur_MS_MV_sing_BS.yaml' 2026-05-11T15:53:38,856 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_plur_BS.yaml' 2026-05-11T15:53:38,857 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__sent_comp__sing_MS_MV_sing_BS.yaml' 2026-05-11T15:53:38,858 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__plur_MS_MV.yaml' 2026-05-11T15:53:38,859 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__simple_agrmt__sing_MS_MV.yaml' 2026-05-11T15:53:38,861 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_plur_ES.yaml' 2026-05-11T15:53:38,862 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__plur_MS_EV_MV_sing_ES.yaml' 2026-05-11T15:53:38,863 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_plur_ES.yaml' 2026-05-11T15:53:38,864 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__subj_rel__sing_MS_EV_MV_sing_ES.yaml' 2026-05-11T15:53:38,865 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__plur_MS_MV_MV.yaml' 2026-05-11T15:53:38,866 adding 'lm_eval/tasks/lm_syneval/lm_syneval__agreement__vp_coord__sing_MS_MV_MV.yaml' 2026-05-11T15:53:38,867 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__future.yaml' 2026-05-11T15:53:38,868 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_anim__past.yaml' 2026-05-11T15:53:38,869 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__future.yaml' 2026-05-11T15:53:38,871 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__npi_across_inanim__past.yaml' 2026-05-11T15:53:38,872 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__future.yaml' 2026-05-11T15:53:38,873 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_anim__past.yaml' 2026-05-11T15:53:38,874 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__future.yaml' 2026-05-11T15:53:38,875 adding 'lm_eval/tasks/lm_syneval/lm_syneval__npi__simple_npi_inanim__past.yaml' 2026-05-11T15:53:38,876 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_plur_BS.yaml' 2026-05-11T15:53:38,877 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__plur_MS_ANPHR_sing_BS.yaml' 2026-05-11T15:53:38,878 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_plur_BS.yaml' 2026-05-11T15:53:38,879 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexive_sent_comp__sing_MS_ANPHR_sing_BS.yaml' 2026-05-11T15:53:38,881 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_plur_ES_EV.yaml' 2026-05-11T15:53:38,882 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__plur_MS_ANPHR_sing_ES_EV.yaml' 2026-05-11T15:53:38,883 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_plur_ES_EV.yaml' 2026-05-11T15:53:38,884 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__reflexives_across__sing_MS_ANPHR_sing_ES_EV.yaml' 2026-05-11T15:53:38,885 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__plur_MS_ANPHR.yaml' 2026-05-11T15:53:38,886 adding 'lm_eval/tasks/lm_syneval/lm_syneval__reflexives__simple_reflexives__sing_MS_ANPHR.yaml' 2026-05-11T15:53:38,888 adding 'lm_eval/tasks/lm_syneval/lm_syneval_group.yaml' 2026-05-11T15:53:38,890 adding 'lm_eval/tasks/logiqa/README.md' 2026-05-11T15:53:38,891 adding 'lm_eval/tasks/logiqa/logiqa.yaml' 2026-05-11T15:53:38,892 adding 'lm_eval/tasks/logiqa/utils_logiqa.py' 2026-05-11T15:53:38,894 adding 'lm_eval/tasks/logiqa2/README.md' 2026-05-11T15:53:38,896 adding 'lm_eval/tasks/logiqa2/logieval.yaml' 2026-05-11T15:53:38,897 adding 'lm_eval/tasks/logiqa2/logiqa2.yaml' 2026-05-11T15:53:38,898 adding 'lm_eval/tasks/logiqa2/utils_logiqa2.py' 2026-05-11T15:53:38,901 adding 'lm_eval/tasks/longbench/2wikimqa.yaml' 2026-05-11T15:53:38,902 adding 'lm_eval/tasks/longbench/2wikimqa_e.yaml' 2026-05-11T15:53:38,904 adding 'lm_eval/tasks/longbench/README.md' 2026-05-11T15:53:38,906 adding 'lm_eval/tasks/longbench/_generate_config.py' 2026-05-11T15:53:38,907 adding 'lm_eval/tasks/longbench/_longbench.yaml' 2026-05-11T15:53:38,909 adding 'lm_eval/tasks/longbench/_longbench_code.yaml' 2026-05-11T15:53:38,910 adding 'lm_eval/tasks/longbench/_longbench_code_e.yaml' 2026-05-11T15:53:38,911 adding 'lm_eval/tasks/longbench/_longbench_e.yaml' 2026-05-11T15:53:38,912 adding 'lm_eval/tasks/longbench/_longbench_fewshot.yaml' 2026-05-11T15:53:38,913 adding 'lm_eval/tasks/longbench/_longbench_fewshot_e.yaml' 2026-05-11T15:53:38,915 adding 'lm_eval/tasks/longbench/_longbench_multi.yaml' 2026-05-11T15:53:38,916 adding 'lm_eval/tasks/longbench/_longbench_multi_e.yaml' 2026-05-11T15:53:38,917 adding 'lm_eval/tasks/longbench/_longbench_single.yaml' 2026-05-11T15:53:38,918 adding 'lm_eval/tasks/longbench/_longbench_single_e.yaml' 2026-05-11T15:53:38,919 adding 'lm_eval/tasks/longbench/_longbench_summarization.yaml' 2026-05-11T15:53:38,920 adding 'lm_eval/tasks/longbench/_longbench_summarization_e.yaml' 2026-05-11T15:53:38,921 adding 'lm_eval/tasks/longbench/_longbench_synthetic.yaml' 2026-05-11T15:53:38,922 adding 'lm_eval/tasks/longbench/_longbench_synthetic_e.yaml' 2026-05-11T15:53:38,924 adding 'lm_eval/tasks/longbench/dureader.yaml' 2026-05-11T15:53:38,925 adding 'lm_eval/tasks/longbench/gov_report.yaml' 2026-05-11T15:53:38,926 adding 'lm_eval/tasks/longbench/gov_report_e.yaml' 2026-05-11T15:53:38,927 adding 'lm_eval/tasks/longbench/hotpotqa.yaml' 2026-05-11T15:53:38,929 adding 'lm_eval/tasks/longbench/hotpotqa_e.yaml' 2026-05-11T15:53:38,930 adding 'lm_eval/tasks/longbench/lcc.yaml' 2026-05-11T15:53:38,931 adding 'lm_eval/tasks/longbench/lcc_e.yaml' 2026-05-11T15:53:38,933 adding 'lm_eval/tasks/longbench/lsht.yaml' 2026-05-11T15:53:38,934 adding 'lm_eval/tasks/longbench/metrics.py' 2026-05-11T15:53:38,936 adding 'lm_eval/tasks/longbench/multi_news.yaml' 2026-05-11T15:53:38,937 adding 'lm_eval/tasks/longbench/multi_news_e.yaml' 2026-05-11T15:53:38,938 adding 'lm_eval/tasks/longbench/multifieldqa_en.yaml' 2026-05-11T15:53:38,939 adding 'lm_eval/tasks/longbench/multifieldqa_en_e.yaml' 2026-05-11T15:53:38,941 adding 'lm_eval/tasks/longbench/multifieldqa_zh.yaml' 2026-05-11T15:53:38,942 adding 'lm_eval/tasks/longbench/musique.yaml' 2026-05-11T15:53:38,943 adding 'lm_eval/tasks/longbench/narrativeqa.yaml' 2026-05-11T15:53:38,945 adding 'lm_eval/tasks/longbench/passage_count.yaml' 2026-05-11T15:53:38,946 adding 'lm_eval/tasks/longbench/passage_count_e.yaml' 2026-05-11T15:53:38,947 adding 'lm_eval/tasks/longbench/passage_retrieval_en.yaml' 2026-05-11T15:53:38,948 adding 'lm_eval/tasks/longbench/passage_retrieval_en_e.yaml' 2026-05-11T15:53:38,950 adding 'lm_eval/tasks/longbench/passage_retrieval_zh.yaml' 2026-05-11T15:53:38,951 adding 'lm_eval/tasks/longbench/qasper.yaml' 2026-05-11T15:53:38,952 adding 'lm_eval/tasks/longbench/qasper_e.yaml' 2026-05-11T15:53:38,953 adding 'lm_eval/tasks/longbench/qmsum.yaml' 2026-05-11T15:53:38,955 adding 'lm_eval/tasks/longbench/repobench-p.yaml' 2026-05-11T15:53:38,956 adding 'lm_eval/tasks/longbench/repobench-p_e.yaml' 2026-05-11T15:53:38,957 adding 'lm_eval/tasks/longbench/samsum.yaml' 2026-05-11T15:53:38,958 adding 'lm_eval/tasks/longbench/samsum_e.yaml' 2026-05-11T15:53:38,960 adding 'lm_eval/tasks/longbench/trec.yaml' 2026-05-11T15:53:38,961 adding 'lm_eval/tasks/longbench/trec_e.yaml' 2026-05-11T15:53:38,962 adding 'lm_eval/tasks/longbench/triviaqa.yaml' 2026-05-11T15:53:38,964 adding 'lm_eval/tasks/longbench/triviaqa_e.yaml' 2026-05-11T15:53:38,965 adding 'lm_eval/tasks/longbench/utils.py' 2026-05-11T15:53:38,966 adding 'lm_eval/tasks/longbench/vcsum.yaml' 2026-05-11T15:53:38,969 adding 'lm_eval/tasks/longbench2/README.md' 2026-05-11T15:53:38,970 adding 'lm_eval/tasks/longbench2/_longbench2.yaml' 2026-05-11T15:53:38,971 adding 'lm_eval/tasks/longbench2/_longbench2_history.yaml' 2026-05-11T15:53:38,973 adding 'lm_eval/tasks/longbench2/_longbench2_incontext.yaml' 2026-05-11T15:53:38,974 adding 'lm_eval/tasks/longbench2/_longbench2_multi.yaml' 2026-05-11T15:53:38,975 adding 'lm_eval/tasks/longbench2/_longbench2_single.yaml' 2026-05-11T15:53:38,976 adding 'lm_eval/tasks/longbench2/_longbench2_structured.yaml' 2026-05-11T15:53:38,977 adding 'lm_eval/tasks/longbench2/_longbench_common_yaml' 2026-05-11T15:53:38,979 adding 'lm_eval/tasks/longbench2/academic_multi_doc.yaml' 2026-05-11T15:53:38,980 adding 'lm_eval/tasks/longbench2/academic_single.yaml' 2026-05-11T15:53:38,981 adding 'lm_eval/tasks/longbench2/agent_history.yaml' 2026-05-11T15:53:38,982 adding 'lm_eval/tasks/longbench2/detective.yaml' 2026-05-11T15:53:38,983 adding 'lm_eval/tasks/longbench2/dialogue_history.yaml' 2026-05-11T15:53:38,984 adding 'lm_eval/tasks/longbench2/event_order.yaml' 2026-05-11T15:53:38,986 adding 'lm_eval/tasks/longbench2/fin_multi_doc.yaml' 2026-05-11T15:53:38,987 adding 'lm_eval/tasks/longbench2/fin_single_doc.yaml' 2026-05-11T15:53:38,988 adding 'lm_eval/tasks/longbench2/govt_multi_doc.yaml' 2026-05-11T15:53:38,989 adding 'lm_eval/tasks/longbench2/govt_single_doc.yaml' 2026-05-11T15:53:38,990 adding 'lm_eval/tasks/longbench2/graph.yaml' 2026-05-11T15:53:38,991 adding 'lm_eval/tasks/longbench2/legal_multi.yaml' 2026-05-11T15:53:38,992 adding 'lm_eval/tasks/longbench2/legal_single.yaml' 2026-05-11T15:53:38,994 adding 'lm_eval/tasks/longbench2/lit_single_doc.yaml' 2026-05-11T15:53:38,995 adding 'lm_eval/tasks/longbench2/longbench2_code.yaml' 2026-05-11T15:53:38,996 adding 'lm_eval/tasks/longbench2/many_shot.yaml' 2026-05-11T15:53:38,997 adding 'lm_eval/tasks/longbench2/news_multi.yaml' 2026-05-11T15:53:38,998 adding 'lm_eval/tasks/longbench2/table.yaml' 2026-05-11T15:53:38,999 adding 'lm_eval/tasks/longbench2/translate.yaml' 2026-05-11T15:53:39,001 adding 'lm_eval/tasks/longbench2/user_guide.yaml' 2026-05-11T15:53:39,003 adding 'lm_eval/tasks/mastermind/README.md' 2026-05-11T15:53:39,004 adding 'lm_eval/tasks/mastermind/mastermind_24_easy.yaml' 2026-05-11T15:53:39,005 adding 'lm_eval/tasks/mastermind/mastermind_24_hard.yaml' 2026-05-11T15:53:39,006 adding 'lm_eval/tasks/mastermind/mastermind_35_easy.yaml' 2026-05-11T15:53:39,007 adding 'lm_eval/tasks/mastermind/mastermind_35_hard.yaml' 2026-05-11T15:53:39,008 adding 'lm_eval/tasks/mastermind/mastermind_46_easy.yaml' 2026-05-11T15:53:39,010 adding 'lm_eval/tasks/mastermind/mastermind_46_hard.yaml' 2026-05-11T15:53:39,011 adding 'lm_eval/tasks/mathqa/README.md' 2026-05-11T15:53:39,013 adding 'lm_eval/tasks/mathqa/mathqa.yaml' 2026-05-11T15:53:39,014 adding 'lm_eval/tasks/mathqa/utils.py' 2026-05-11T15:53:39,016 adding 'lm_eval/tasks/mbpp/README.md' 2026-05-11T15:53:39,017 adding 'lm_eval/tasks/mbpp/mbpp.yaml' 2026-05-11T15:53:39,018 adding 'lm_eval/tasks/mbpp/mbpp_instruct.yaml' 2026-05-11T15:53:39,019 adding 'lm_eval/tasks/mbpp/mbpp_plus.yaml' 2026-05-11T15:53:39,021 adding 'lm_eval/tasks/mbpp/mbpp_plus_instruct.yaml' 2026-05-11T15:53:39,022 adding 'lm_eval/tasks/mbpp/utils.py' 2026-05-11T15:53:39,024 adding 'lm_eval/tasks/mc_taco/README.md' 2026-05-11T15:53:39,025 adding 'lm_eval/tasks/mc_taco/default.yaml' 2026-05-11T15:53:39,027 adding 'lm_eval/tasks/med_concepts_qa/README.md' 2026-05-11T15:53:39,029 adding 'lm_eval/tasks/med_concepts_qa/_default_template_yaml' 2026-05-11T15:53:39,030 adding 'lm_eval/tasks/med_concepts_qa/_generate_configs.py' 2026-05-11T15:53:39,031 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa.yaml' 2026-05-11T15:53:39,032 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_atc.yaml' 2026-05-11T15:53:39,033 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10cm.yaml' 2026-05-11T15:53:39,035 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd10proc.yaml' 2026-05-11T15:53:39,036 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9cm.yaml' 2026-05-11T15:53:39,037 adding 'lm_eval/tasks/med_concepts_qa/_med_concepts_qa_icd9proc.yaml' 2026-05-11T15:53:39,038 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_easy.yaml' 2026-05-11T15:53:39,039 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_hard.yaml' 2026-05-11T15:53:39,040 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_atc_medium.yaml' 2026-05-11T15:53:39,042 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_easy.yaml' 2026-05-11T15:53:39,043 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_hard.yaml' 2026-05-11T15:53:39,044 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10cm_medium.yaml' 2026-05-11T15:53:39,045 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_easy.yaml' 2026-05-11T15:53:39,046 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_hard.yaml' 2026-05-11T15:53:39,048 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd10proc_medium.yaml' 2026-05-11T15:53:39,049 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_easy.yaml' 2026-05-11T15:53:39,050 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_hard.yaml' 2026-05-11T15:53:39,051 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9cm_medium.yaml' 2026-05-11T15:53:39,052 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_easy.yaml' 2026-05-11T15:53:39,053 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_hard.yaml' 2026-05-11T15:53:39,054 adding 'lm_eval/tasks/med_concepts_qa/med_concepts_qa_icd9proc_medium.yaml' 2026-05-11T15:53:39,056 adding 'lm_eval/tasks/med_prescriptions/med_prescriptions_easy.yaml' 2026-05-11T15:53:39,057 adding 'lm_eval/tasks/med_prescriptions/med_prescriptions_hard.yaml' 2026-05-11T15:53:39,068 adding 'lm_eval/tasks/med_prescriptions/utils.py' 2026-05-11T15:53:39,070 adding 'lm_eval/tasks/med_text_classification/med_text_classification_easy.yaml' 2026-05-11T15:53:39,072 adding 'lm_eval/tasks/med_text_classification/med_text_classification_hard.yaml' 2026-05-11T15:53:39,073 adding 'lm_eval/tasks/med_text_classification/utils.py' 2026-05-11T15:53:39,075 adding 'lm_eval/tasks/meddialog/README.md' 2026-05-11T15:53:39,076 adding 'lm_eval/tasks/meddialog/meddialog_qsumm.yaml' 2026-05-11T15:53:39,077 adding 'lm_eval/tasks/meddialog/meddialog_qsumm_perplexity.yaml' 2026-05-11T15:53:39,079 adding 'lm_eval/tasks/meddialog/meddialog_raw_dialogues.yaml' 2026-05-11T15:53:39,080 adding 'lm_eval/tasks/meddialog/meddialog_raw_perplexity.yaml' 2026-05-11T15:53:39,081 adding 'lm_eval/tasks/meddialog/utils.py' 2026-05-11T15:53:39,082 adding 'lm_eval/tasks/meddialog/utils_perplexity.py' 2026-05-11T15:53:39,084 adding 'lm_eval/tasks/mediqa_qa2019/README.md' 2026-05-11T15:53:39,086 adding 'lm_eval/tasks/mediqa_qa2019/mediqa_qa2019.yaml' 2026-05-11T15:53:39,087 adding 'lm_eval/tasks/mediqa_qa2019/mediqa_qa2019_perplexity.yaml' 2026-05-11T15:53:39,089 adding 'lm_eval/tasks/mediqa_qa2019/utils.py' 2026-05-11T15:53:39,090 adding 'lm_eval/tasks/mediqa_qa2019/utils_perplexity.py' 2026-05-11T15:53:39,091 adding 'lm_eval/tasks/medmcqa/medmcqa.yaml' 2026-05-11T15:53:39,093 adding 'lm_eval/tasks/medmcqa/utils_medmcqa.py' 2026-05-11T15:53:39,095 adding 'lm_eval/tasks/medqa/medqa.yaml' 2026-05-11T15:53:39,096 adding 'lm_eval/tasks/medqa/preprocess_medqa.py' 2026-05-11T15:53:39,098 adding 'lm_eval/tasks/medtext/README.md' 2026-05-11T15:53:39,099 adding 'lm_eval/tasks/medtext/medtext.yaml' 2026-05-11T15:53:39,100 adding 'lm_eval/tasks/medtext/medtext_perplexity.yaml' 2026-05-11T15:53:39,102 adding 'lm_eval/tasks/medtext/utils.py' 2026-05-11T15:53:39,103 adding 'lm_eval/tasks/medtext/utils_perplexity.py' 2026-05-11T15:53:39,105 adding 'lm_eval/tasks/mela/README.md' 2026-05-11T15:53:39,106 adding 'lm_eval/tasks/mela/_mela.yaml' 2026-05-11T15:53:39,108 adding 'lm_eval/tasks/mela/mela_ar.yaml' 2026-05-11T15:53:39,109 adding 'lm_eval/tasks/mela/mela_de.yaml' 2026-05-11T15:53:39,110 adding 'lm_eval/tasks/mela/mela_en.yaml' 2026-05-11T15:53:39,111 adding 'lm_eval/tasks/mela/mela_es.yaml' 2026-05-11T15:53:39,113 adding 'lm_eval/tasks/mela/mela_fr.yaml' 2026-05-11T15:53:39,114 adding 'lm_eval/tasks/mela/mela_is.yaml' 2026-05-11T15:53:39,115 adding 'lm_eval/tasks/mela/mela_it.yaml' 2026-05-11T15:53:39,116 adding 'lm_eval/tasks/mela/mela_ja.yaml' 2026-05-11T15:53:39,118 adding 'lm_eval/tasks/mela/mela_ru.yaml' 2026-05-11T15:53:39,119 adding 'lm_eval/tasks/mela/mela_zh.yaml' 2026-05-11T15:53:39,121 adding 'lm_eval/tasks/meqsum/README.md' 2026-05-11T15:53:39,122 adding 'lm_eval/tasks/meqsum/meqsum.yaml' 2026-05-11T15:53:39,123 adding 'lm_eval/tasks/meqsum/utils.py' 2026-05-11T15:53:39,126 adding 'lm_eval/tasks/metabench/README.md' 2026-05-11T15:53:39,128 adding 'lm_eval/tasks/metabench/metabench.yaml' 2026-05-11T15:53:39,129 adding 'lm_eval/tasks/metabench/metabench_arc.yaml' 2026-05-11T15:53:39,130 adding 'lm_eval/tasks/metabench/metabench_arc_permute.yaml' 2026-05-11T15:53:39,131 adding 'lm_eval/tasks/metabench/metabench_arc_secondary.yaml' 2026-05-11T15:53:39,132 adding 'lm_eval/tasks/metabench/metabench_arc_secondary_permute.yaml' 2026-05-11T15:53:39,133 adding 'lm_eval/tasks/metabench/metabench_gsm8k.yaml' 2026-05-11T15:53:39,134 adding 'lm_eval/tasks/metabench/metabench_gsm8k_secondary.yaml' 2026-05-11T15:53:39,135 adding 'lm_eval/tasks/metabench/metabench_hellaswag.yaml' 2026-05-11T15:53:39,137 adding 'lm_eval/tasks/metabench/metabench_hellaswag_permute.yaml' 2026-05-11T15:53:39,138 adding 'lm_eval/tasks/metabench/metabench_hellaswag_secondary.yaml' 2026-05-11T15:53:39,139 adding 'lm_eval/tasks/metabench/metabench_hellaswag_secondary_permute.yaml' 2026-05-11T15:53:39,140 adding 'lm_eval/tasks/metabench/metabench_mmlu.yaml' 2026-05-11T15:53:39,141 adding 'lm_eval/tasks/metabench/metabench_mmlu_permute.yaml' 2026-05-11T15:53:39,142 adding 'lm_eval/tasks/metabench/metabench_mmlu_secondary.yaml' 2026-05-11T15:53:39,144 adding 'lm_eval/tasks/metabench/metabench_mmlu_secondary_permute.yaml' 2026-05-11T15:53:39,145 adding 'lm_eval/tasks/metabench/metabench_permute.yaml' 2026-05-11T15:53:39,146 adding 'lm_eval/tasks/metabench/metabench_secondary.yaml' 2026-05-11T15:53:39,147 adding 'lm_eval/tasks/metabench/metabench_secondary_permute.yaml' 2026-05-11T15:53:39,148 adding 'lm_eval/tasks/metabench/metabench_truthfulqa.yaml' 2026-05-11T15:53:39,149 adding 'lm_eval/tasks/metabench/metabench_truthfulqa_permute.yaml' 2026-05-11T15:53:39,150 adding 'lm_eval/tasks/metabench/metabench_truthfulqa_secondary.yaml' 2026-05-11T15:53:39,152 adding 'lm_eval/tasks/metabench/metabench_truthfulqa_secondary_permute.yaml' 2026-05-11T15:53:39,153 adding 'lm_eval/tasks/metabench/metabench_winogrande.yaml' 2026-05-11T15:53:39,154 adding 'lm_eval/tasks/metabench/metabench_winogrande_permute.yaml' 2026-05-11T15:53:39,155 adding 'lm_eval/tasks/metabench/metabench_winogrande_secondary.yaml' 2026-05-11T15:53:39,156 adding 'lm_eval/tasks/metabench/metabench_winogrande_secondary_permute.yaml' 2026-05-11T15:53:39,158 adding 'lm_eval/tasks/metabench/process_docs.py' 2026-05-11T15:53:39,160 adding 'lm_eval/tasks/metabench/process_docs_permute.py' 2026-05-11T15:53:39,162 adding 'lm_eval/tasks/mgsm/README.md' 2026-05-11T15:53:39,163 adding 'lm_eval/tasks/mgsm/gen_yaml.sh' 2026-05-11T15:53:39,165 adding 'lm_eval/tasks/mgsm/utils.py' 2026-05-11T15:53:39,167 adding 'lm_eval/tasks/mgsm/direct/direct_yaml' 2026-05-11T15:53:39,168 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_bn.yaml' 2026-05-11T15:53:39,169 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_de.yaml' 2026-05-11T15:53:39,170 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_en.yaml' 2026-05-11T15:53:39,172 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_es.yaml' 2026-05-11T15:53:39,173 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_fr.yaml' 2026-05-11T15:53:39,174 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ja.yaml' 2026-05-11T15:53:39,175 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_ru.yaml' 2026-05-11T15:53:39,176 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_sw.yaml' 2026-05-11T15:53:39,178 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_te.yaml' 2026-05-11T15:53:39,179 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_th.yaml' 2026-05-11T15:53:39,180 adding 'lm_eval/tasks/mgsm/direct/mgsm_direct_zh.yaml' 2026-05-11T15:53:39,182 adding 'lm_eval/tasks/mgsm/en_cot/cot_yaml' 2026-05-11T15:53:39,183 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_bn.yaml' 2026-05-11T15:53:39,185 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_de.yaml' 2026-05-11T15:53:39,186 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_en.yaml' 2026-05-11T15:53:39,187 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_es.yaml' 2026-05-11T15:53:39,189 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_fr.yaml' 2026-05-11T15:53:39,190 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ja.yaml' 2026-05-11T15:53:39,191 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_ru.yaml' 2026-05-11T15:53:39,192 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_sw.yaml' 2026-05-11T15:53:39,193 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_te.yaml' 2026-05-11T15:53:39,194 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_th.yaml' 2026-05-11T15:53:39,195 adding 'lm_eval/tasks/mgsm/en_cot/mgsm_en_cot_zh.yaml' 2026-05-11T15:53:39,197 adding 'lm_eval/tasks/mgsm/native_cot/cot_yaml' 2026-05-11T15:53:39,198 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_bn.yaml' 2026-05-11T15:53:39,199 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_de.yaml' 2026-05-11T15:53:39,201 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_en.yaml' 2026-05-11T15:53:39,202 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_es.yaml' 2026-05-11T15:53:39,203 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_fr.yaml' 2026-05-11T15:53:39,204 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ja.yaml' 2026-05-11T15:53:39,205 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_ru.yaml' 2026-05-11T15:53:39,207 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_sw.yaml' 2026-05-11T15:53:39,208 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_te.yaml' 2026-05-11T15:53:39,209 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_th.yaml' 2026-05-11T15:53:39,210 adding 'lm_eval/tasks/mgsm/native_cot/mgsm_native_cot_zh.yaml' 2026-05-11T15:53:39,212 adding 'lm_eval/tasks/mimic_repsum/README.md' 2026-05-11T15:53:39,214 adding 'lm_eval/tasks/mimic_repsum/mimic_repsum.yaml' 2026-05-11T15:53:39,215 adding 'lm_eval/tasks/mimic_repsum/mimic_repsum_perplexity.yaml' 2026-05-11T15:53:39,216 adding 'lm_eval/tasks/mimic_repsum/utils.py' 2026-05-11T15:53:39,217 adding 'lm_eval/tasks/mimic_repsum/utils_perplexity.py' 2026-05-11T15:53:39,219 adding 'lm_eval/tasks/minerva_math/README.md' 2026-05-11T15:53:39,221 adding 'lm_eval/tasks/minerva_math/minerva_math500.yaml' 2026-05-11T15:53:39,222 adding 'lm_eval/tasks/minerva_math/minerva_math_algebra.yaml' 2026-05-11T15:53:39,223 adding 'lm_eval/tasks/minerva_math/minerva_math_counting_and_prob.yaml' 2026-05-11T15:53:39,224 adding 'lm_eval/tasks/minerva_math/minerva_math_geometry.yaml' 2026-05-11T15:53:39,225 adding 'lm_eval/tasks/minerva_math/minerva_math_intermediate_algebra.yaml' 2026-05-11T15:53:39,226 adding 'lm_eval/tasks/minerva_math/minerva_math_num_theory.yaml' 2026-05-11T15:53:39,228 adding 'lm_eval/tasks/minerva_math/minerva_math_prealgebra.yaml' 2026-05-11T15:53:39,229 adding 'lm_eval/tasks/minerva_math/minerva_math_precalc.yaml' 2026-05-11T15:53:39,231 adding 'lm_eval/tasks/minerva_math/utils.py' 2026-05-11T15:53:39,234 adding 'lm_eval/tasks/mlqa/README.md' 2026-05-11T15:53:39,235 adding 'lm_eval/tasks/mlqa/generate_tasks.py' 2026-05-11T15:53:39,236 adding 'lm_eval/tasks/mlqa/mlqa_ar_ar.yaml' 2026-05-11T15:53:39,238 adding 'lm_eval/tasks/mlqa/mlqa_ar_de.yaml' 2026-05-11T15:53:39,239 adding 'lm_eval/tasks/mlqa/mlqa_ar_en.yaml' 2026-05-11T15:53:39,240 adding 'lm_eval/tasks/mlqa/mlqa_ar_es.yaml' 2026-05-11T15:53:39,241 adding 'lm_eval/tasks/mlqa/mlqa_ar_hi.yaml' 2026-05-11T15:53:39,242 adding 'lm_eval/tasks/mlqa/mlqa_ar_vi.yaml' 2026-05-11T15:53:39,244 adding 'lm_eval/tasks/mlqa/mlqa_ar_zh.yaml' 2026-05-11T15:53:39,245 adding 'lm_eval/tasks/mlqa/mlqa_common_yaml' 2026-05-11T15:53:39,246 adding 'lm_eval/tasks/mlqa/mlqa_de_ar.yaml' 2026-05-11T15:53:39,248 adding 'lm_eval/tasks/mlqa/mlqa_de_de.yaml' 2026-05-11T15:53:39,249 adding 'lm_eval/tasks/mlqa/mlqa_de_en.yaml' 2026-05-11T15:53:39,250 adding 'lm_eval/tasks/mlqa/mlqa_de_es.yaml' 2026-05-11T15:53:39,251 adding 'lm_eval/tasks/mlqa/mlqa_de_hi.yaml' 2026-05-11T15:53:39,252 adding 'lm_eval/tasks/mlqa/mlqa_de_vi.yaml' 2026-05-11T15:53:39,254 adding 'lm_eval/tasks/mlqa/mlqa_de_zh.yaml' 2026-05-11T15:53:39,255 adding 'lm_eval/tasks/mlqa/mlqa_en_ar.yaml' 2026-05-11T15:53:39,256 adding 'lm_eval/tasks/mlqa/mlqa_en_de.yaml' 2026-05-11T15:53:39,257 adding 'lm_eval/tasks/mlqa/mlqa_en_en.yaml' 2026-05-11T15:53:39,258 adding 'lm_eval/tasks/mlqa/mlqa_en_es.yaml' 2026-05-11T15:53:39,259 adding 'lm_eval/tasks/mlqa/mlqa_en_hi.yaml' 2026-05-11T15:53:39,260 adding 'lm_eval/tasks/mlqa/mlqa_en_vi.yaml' 2026-05-11T15:53:39,262 adding 'lm_eval/tasks/mlqa/mlqa_en_zh.yaml' 2026-05-11T15:53:39,263 adding 'lm_eval/tasks/mlqa/mlqa_es_ar.yaml' 2026-05-11T15:53:39,264 adding 'lm_eval/tasks/mlqa/mlqa_es_de.yaml' 2026-05-11T15:53:39,265 adding 'lm_eval/tasks/mlqa/mlqa_es_en.yaml' 2026-05-11T15:53:39,266 adding 'lm_eval/tasks/mlqa/mlqa_es_es.yaml' 2026-05-11T15:53:39,267 adding 'lm_eval/tasks/mlqa/mlqa_es_hi.yaml' 2026-05-11T15:53:39,268 adding 'lm_eval/tasks/mlqa/mlqa_es_vi.yaml' 2026-05-11T15:53:39,269 adding 'lm_eval/tasks/mlqa/mlqa_es_zh.yaml' 2026-05-11T15:53:39,271 adding 'lm_eval/tasks/mlqa/mlqa_hi_ar.yaml' 2026-05-11T15:53:39,272 adding 'lm_eval/tasks/mlqa/mlqa_hi_de.yaml' 2026-05-11T15:53:39,273 adding 'lm_eval/tasks/mlqa/mlqa_hi_en.yaml' 2026-05-11T15:53:39,274 adding 'lm_eval/tasks/mlqa/mlqa_hi_es.yaml' 2026-05-11T15:53:39,275 adding 'lm_eval/tasks/mlqa/mlqa_hi_hi.yaml' 2026-05-11T15:53:39,276 adding 'lm_eval/tasks/mlqa/mlqa_hi_vi.yaml' 2026-05-11T15:53:39,277 adding 'lm_eval/tasks/mlqa/mlqa_hi_zh.yaml' 2026-05-11T15:53:39,279 adding 'lm_eval/tasks/mlqa/mlqa_vi_ar.yaml' 2026-05-11T15:53:39,280 adding 'lm_eval/tasks/mlqa/mlqa_vi_de.yaml' 2026-05-11T15:53:39,281 adding 'lm_eval/tasks/mlqa/mlqa_vi_en.yaml' 2026-05-11T15:53:39,282 adding 'lm_eval/tasks/mlqa/mlqa_vi_es.yaml' 2026-05-11T15:53:39,283 adding 'lm_eval/tasks/mlqa/mlqa_vi_hi.yaml' 2026-05-11T15:53:39,284 adding 'lm_eval/tasks/mlqa/mlqa_vi_vi.yaml' 2026-05-11T15:53:39,285 adding 'lm_eval/tasks/mlqa/mlqa_vi_zh.yaml' 2026-05-11T15:53:39,287 adding 'lm_eval/tasks/mlqa/mlqa_zh_ar.yaml' 2026-05-11T15:53:39,288 adding 'lm_eval/tasks/mlqa/mlqa_zh_de.yaml' 2026-05-11T15:53:39,289 adding 'lm_eval/tasks/mlqa/mlqa_zh_en.yaml' 2026-05-11T15:53:39,290 adding 'lm_eval/tasks/mlqa/mlqa_zh_es.yaml' 2026-05-11T15:53:39,291 adding 'lm_eval/tasks/mlqa/mlqa_zh_hi.yaml' 2026-05-11T15:53:39,292 adding 'lm_eval/tasks/mlqa/mlqa_zh_vi.yaml' 2026-05-11T15:53:39,294 adding 'lm_eval/tasks/mlqa/mlqa_zh_zh.yaml' 2026-05-11T15:53:39,295 adding 'lm_eval/tasks/mlqa/utils.py' 2026-05-11T15:53:39,297 adding 'lm_eval/tasks/mmlu/README.md' 2026-05-11T15:53:39,299 adding 'lm_eval/tasks/mmlu/_generate_configs.py' 2026-05-11T15:53:39,302 adding 'lm_eval/tasks/mmlu/continuation/_continuation_template_yaml' 2026-05-11T15:53:39,303 adding 'lm_eval/tasks/mmlu/continuation/_mmlu.yaml' 2026-05-11T15:53:39,304 adding 'lm_eval/tasks/mmlu/continuation/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,306 adding 'lm_eval/tasks/mmlu/continuation/mmlu_anatomy.yaml' 2026-05-11T15:53:39,307 adding 'lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml' 2026-05-11T15:53:39,308 adding 'lm_eval/tasks/mmlu/continuation/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,309 adding 'lm_eval/tasks/mmlu/continuation/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,311 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_biology.yaml' 2026-05-11T15:53:39,312 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,313 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,314 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,315 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,317 adding 'lm_eval/tasks/mmlu/continuation/mmlu_college_physics.yaml' 2026-05-11T15:53:39,318 adding 'lm_eval/tasks/mmlu/continuation/mmlu_computer_security.yaml' 2026-05-11T15:53:39,319 adding 'lm_eval/tasks/mmlu/continuation/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,320 adding 'lm_eval/tasks/mmlu/continuation/mmlu_econometrics.yaml' 2026-05-11T15:53:39,322 adding 'lm_eval/tasks/mmlu/continuation/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,323 adding 'lm_eval/tasks/mmlu/continuation/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,324 adding 'lm_eval/tasks/mmlu/continuation/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,325 adding 'lm_eval/tasks/mmlu/continuation/mmlu_global_facts.yaml' 2026-05-11T15:53:39,326 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,327 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,329 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,330 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,331 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,332 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,333 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,335 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,336 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,337 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,339 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,342 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,343 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,344 adding 'lm_eval/tasks/mmlu/continuation/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,346 adding 'lm_eval/tasks/mmlu/continuation/mmlu_human_aging.yaml' 2026-05-11T15:53:39,347 adding 'lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,348 adding 'lm_eval/tasks/mmlu/continuation/mmlu_international_law.yaml' 2026-05-11T15:53:39,349 adding 'lm_eval/tasks/mmlu/continuation/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,350 adding 'lm_eval/tasks/mmlu/continuation/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,351 adding 'lm_eval/tasks/mmlu/continuation/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,353 adding 'lm_eval/tasks/mmlu/continuation/mmlu_management.yaml' 2026-05-11T15:53:39,354 adding 'lm_eval/tasks/mmlu/continuation/mmlu_marketing.yaml' 2026-05-11T15:53:39,355 adding 'lm_eval/tasks/mmlu/continuation/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,356 adding 'lm_eval/tasks/mmlu/continuation/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,357 adding 'lm_eval/tasks/mmlu/continuation/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,359 adding 'lm_eval/tasks/mmlu/continuation/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,360 adding 'lm_eval/tasks/mmlu/continuation/mmlu_nutrition.yaml' 2026-05-11T15:53:39,361 adding 'lm_eval/tasks/mmlu/continuation/mmlu_philosophy.yaml' 2026-05-11T15:53:39,362 adding 'lm_eval/tasks/mmlu/continuation/mmlu_prehistory.yaml' 2026-05-11T15:53:39,363 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,364 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_law.yaml' 2026-05-11T15:53:39,366 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,367 adding 'lm_eval/tasks/mmlu/continuation/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,368 adding 'lm_eval/tasks/mmlu/continuation/mmlu_public_relations.yaml' 2026-05-11T15:53:39,369 adding 'lm_eval/tasks/mmlu/continuation/mmlu_security_studies.yaml' 2026-05-11T15:53:39,370 adding 'lm_eval/tasks/mmlu/continuation/mmlu_sociology.yaml' 2026-05-11T15:53:39,372 adding 'lm_eval/tasks/mmlu/continuation/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,373 adding 'lm_eval/tasks/mmlu/continuation/mmlu_virology.yaml' 2026-05-11T15:53:39,374 adding 'lm_eval/tasks/mmlu/continuation/mmlu_world_religions.yaml' 2026-05-11T15:53:39,377 adding 'lm_eval/tasks/mmlu/default/_default_template_yaml' 2026-05-11T15:53:39,378 adding 'lm_eval/tasks/mmlu/default/_mmlu.yaml' 2026-05-11T15:53:39,379 adding 'lm_eval/tasks/mmlu/default/_mmlu_humanities.yaml' 2026-05-11T15:53:39,381 adding 'lm_eval/tasks/mmlu/default/_mmlu_other.yaml' 2026-05-11T15:53:39,382 adding 'lm_eval/tasks/mmlu/default/_mmlu_social_sciences.yaml' 2026-05-11T15:53:39,383 adding 'lm_eval/tasks/mmlu/default/_mmlu_stem.yaml' 2026-05-11T15:53:39,384 adding 'lm_eval/tasks/mmlu/default/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,385 adding 'lm_eval/tasks/mmlu/default/mmlu_anatomy.yaml' 2026-05-11T15:53:39,386 adding 'lm_eval/tasks/mmlu/default/mmlu_astronomy.yaml' 2026-05-11T15:53:39,387 adding 'lm_eval/tasks/mmlu/default/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,389 adding 'lm_eval/tasks/mmlu/default/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,390 adding 'lm_eval/tasks/mmlu/default/mmlu_college_biology.yaml' 2026-05-11T15:53:39,391 adding 'lm_eval/tasks/mmlu/default/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,392 adding 'lm_eval/tasks/mmlu/default/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,393 adding 'lm_eval/tasks/mmlu/default/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,395 adding 'lm_eval/tasks/mmlu/default/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,396 adding 'lm_eval/tasks/mmlu/default/mmlu_college_physics.yaml' 2026-05-11T15:53:39,397 adding 'lm_eval/tasks/mmlu/default/mmlu_computer_security.yaml' 2026-05-11T15:53:39,398 adding 'lm_eval/tasks/mmlu/default/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,399 adding 'lm_eval/tasks/mmlu/default/mmlu_econometrics.yaml' 2026-05-11T15:53:39,400 adding 'lm_eval/tasks/mmlu/default/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,401 adding 'lm_eval/tasks/mmlu/default/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,403 adding 'lm_eval/tasks/mmlu/default/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,404 adding 'lm_eval/tasks/mmlu/default/mmlu_global_facts.yaml' 2026-05-11T15:53:39,405 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,406 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,407 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,409 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,410 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,411 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,412 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,413 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,415 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,416 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,417 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,419 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,420 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,421 adding 'lm_eval/tasks/mmlu/default/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,422 adding 'lm_eval/tasks/mmlu/default/mmlu_human_aging.yaml' 2026-05-11T15:53:39,423 adding 'lm_eval/tasks/mmlu/default/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,425 adding 'lm_eval/tasks/mmlu/default/mmlu_international_law.yaml' 2026-05-11T15:53:39,426 adding 'lm_eval/tasks/mmlu/default/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,427 adding 'lm_eval/tasks/mmlu/default/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,428 adding 'lm_eval/tasks/mmlu/default/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,430 adding 'lm_eval/tasks/mmlu/default/mmlu_management.yaml' 2026-05-11T15:53:39,431 adding 'lm_eval/tasks/mmlu/default/mmlu_marketing.yaml' 2026-05-11T15:53:39,432 adding 'lm_eval/tasks/mmlu/default/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,433 adding 'lm_eval/tasks/mmlu/default/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,434 adding 'lm_eval/tasks/mmlu/default/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,435 adding 'lm_eval/tasks/mmlu/default/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,436 adding 'lm_eval/tasks/mmlu/default/mmlu_nutrition.yaml' 2026-05-11T15:53:39,437 adding 'lm_eval/tasks/mmlu/default/mmlu_philosophy.yaml' 2026-05-11T15:53:39,439 adding 'lm_eval/tasks/mmlu/default/mmlu_prehistory.yaml' 2026-05-11T15:53:39,440 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,441 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_law.yaml' 2026-05-11T15:53:39,442 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,443 adding 'lm_eval/tasks/mmlu/default/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,444 adding 'lm_eval/tasks/mmlu/default/mmlu_public_relations.yaml' 2026-05-11T15:53:39,446 adding 'lm_eval/tasks/mmlu/default/mmlu_security_studies.yaml' 2026-05-11T15:53:39,447 adding 'lm_eval/tasks/mmlu/default/mmlu_sociology.yaml' 2026-05-11T15:53:39,448 adding 'lm_eval/tasks/mmlu/default/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,449 adding 'lm_eval/tasks/mmlu/default/mmlu_virology.yaml' 2026-05-11T15:53:39,450 adding 'lm_eval/tasks/mmlu/default/mmlu_world_religions.yaml' 2026-05-11T15:53:39,490 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_cot_prompts.json' 2026-05-11T15:53:39,493 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu.yaml' 2026-05-11T15:53:39,494 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/_mmlu_flan_cot_fewshot_template_yaml' 2026-05-11T15:53:39,496 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,497 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_anatomy.yaml' 2026-05-11T15:53:39,499 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_astronomy.yaml' 2026-05-11T15:53:39,500 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,502 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,503 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml' 2026-05-11T15:53:39,505 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,506 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,508 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,510 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,511 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml' 2026-05-11T15:53:39,513 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml' 2026-05-11T15:53:39,514 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,516 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml' 2026-05-11T15:53:39,517 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,519 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,520 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,522 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml' 2026-05-11T15:53:39,523 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,525 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,527 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,529 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,531 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,532 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,534 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,535 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,536 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,538 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,539 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,541 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,543 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,545 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,546 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml' 2026-05-11T15:53:39,547 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,549 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml' 2026-05-11T15:53:39,550 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,552 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,554 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,555 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml' 2026-05-11T15:53:39,556 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml' 2026-05-11T15:53:39,558 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,559 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,561 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,562 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,563 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml' 2026-05-11T15:53:39,565 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml' 2026-05-11T15:53:39,566 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml' 2026-05-11T15:53:39,568 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,570 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml' 2026-05-11T15:53:39,571 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,573 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,574 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml' 2026-05-11T15:53:39,576 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml' 2026-05-11T15:53:39,578 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml' 2026-05-11T15:53:39,579 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,580 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_virology.yaml' 2026-05-11T15:53:39,582 adding 'lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_world_religions.yaml' 2026-05-11T15:53:39,585 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu.yaml' 2026-05-11T15:53:39,586 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/_mmlu_flan_cot_zeroshot_template_yaml' 2026-05-11T15:53:39,588 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,589 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_anatomy.yaml' 2026-05-11T15:53:39,590 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_astronomy.yaml' 2026-05-11T15:53:39,591 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,593 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,594 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_biology.yaml' 2026-05-11T15:53:39,596 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,597 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,598 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,599 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,600 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_college_physics.yaml' 2026-05-11T15:53:39,601 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_computer_security.yaml' 2026-05-11T15:53:39,603 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,604 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_econometrics.yaml' 2026-05-11T15:53:39,605 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,606 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,608 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,609 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_global_facts.yaml' 2026-05-11T15:53:39,610 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,611 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,613 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,614 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,615 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,616 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,617 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,619 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,620 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,621 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,622 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,624 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,625 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,626 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,627 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_aging.yaml' 2026-05-11T15:53:39,629 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,630 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_international_law.yaml' 2026-05-11T15:53:39,631 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,632 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,633 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,635 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml' 2026-05-11T15:53:39,636 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml' 2026-05-11T15:53:39,637 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,638 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,639 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,641 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,642 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml' 2026-05-11T15:53:39,643 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml' 2026-05-11T15:53:39,645 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml' 2026-05-11T15:53:39,646 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,647 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml' 2026-05-11T15:53:39,648 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,649 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,650 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml' 2026-05-11T15:53:39,651 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml' 2026-05-11T15:53:39,652 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml' 2026-05-11T15:53:39,654 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,655 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml' 2026-05-11T15:53:39,656 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml' 2026-05-11T15:53:39,657 adding 'lm_eval/tasks/mmlu/flan_cot_zeroshot/utils.py' 2026-05-11T15:53:39,661 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml' 2026-05-11T15:53:39,662 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml' 2026-05-11T15:53:39,663 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,664 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml' 2026-05-11T15:53:39,665 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml' 2026-05-11T15:53:39,667 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,668 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,669 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml' 2026-05-11T15:53:39,670 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,672 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,673 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,674 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,675 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_physics.yaml' 2026-05-11T15:53:39,676 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_computer_security.yaml' 2026-05-11T15:53:39,677 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,678 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_econometrics.yaml' 2026-05-11T15:53:39,680 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,681 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,682 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,683 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_global_facts.yaml' 2026-05-11T15:53:39,685 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,686 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,687 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,688 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,689 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,691 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,692 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,693 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,694 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,695 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,697 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,698 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,699 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,700 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,701 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml' 2026-05-11T15:53:39,702 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,703 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml' 2026-05-11T15:53:39,705 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,706 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,707 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,708 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml' 2026-05-11T15:53:39,709 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml' 2026-05-11T15:53:39,711 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,712 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,713 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,714 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,715 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml' 2026-05-11T15:53:39,717 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml' 2026-05-11T15:53:39,718 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml' 2026-05-11T15:53:39,719 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,720 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_law.yaml' 2026-05-11T15:53:39,721 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,722 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,724 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_public_relations.yaml' 2026-05-11T15:53:39,725 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_security_studies.yaml' 2026-05-11T15:53:39,726 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_sociology.yaml' 2026-05-11T15:53:39,728 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,729 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_virology.yaml' 2026-05-11T15:53:39,731 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_world_religions.yaml' 2026-05-11T15:53:39,732 adding 'lm_eval/tasks/mmlu/flan_n_shot/generative/utils.py' 2026-05-11T15:53:39,735 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu.yaml' 2026-05-11T15:53:39,737 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/_mmlu_flan_loglikelihood_template_yaml' 2026-05-11T15:53:39,738 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,740 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_anatomy.yaml' 2026-05-11T15:53:39,741 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_astronomy.yaml' 2026-05-11T15:53:39,742 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,743 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,745 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_biology.yaml' 2026-05-11T15:53:39,746 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,747 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,748 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,749 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,750 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_college_physics.yaml' 2026-05-11T15:53:39,752 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_computer_security.yaml' 2026-05-11T15:53:39,753 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,754 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_econometrics.yaml' 2026-05-11T15:53:39,755 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,756 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,758 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,759 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_global_facts.yaml' 2026-05-11T15:53:39,760 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,761 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,762 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,763 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,764 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,766 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,767 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,768 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,769 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,770 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,772 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,773 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,774 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,775 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,776 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_aging.yaml' 2026-05-11T15:53:39,778 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,779 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_international_law.yaml' 2026-05-11T15:53:39,780 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,781 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,782 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,784 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_management.yaml' 2026-05-11T15:53:39,785 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_marketing.yaml' 2026-05-11T15:53:39,786 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,787 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,789 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,790 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,791 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_nutrition.yaml' 2026-05-11T15:53:39,792 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_philosophy.yaml' 2026-05-11T15:53:39,793 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_prehistory.yaml' 2026-05-11T15:53:39,794 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,795 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_law.yaml' 2026-05-11T15:53:39,797 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,798 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,799 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_public_relations.yaml' 2026-05-11T15:53:39,800 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_security_studies.yaml' 2026-05-11T15:53:39,801 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_sociology.yaml' 2026-05-11T15:53:39,803 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,804 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_virology.yaml' 2026-05-11T15:53:39,805 adding 'lm_eval/tasks/mmlu/flan_n_shot/loglikelihood/mmlu_world_religions.yaml' 2026-05-11T15:53:39,808 adding 'lm_eval/tasks/mmlu/generative/_default_template_yaml' 2026-05-11T15:53:39,809 adding 'lm_eval/tasks/mmlu/generative/_mmlu.yaml' 2026-05-11T15:53:39,810 adding 'lm_eval/tasks/mmlu/generative/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,811 adding 'lm_eval/tasks/mmlu/generative/mmlu_anatomy.yaml' 2026-05-11T15:53:39,812 adding 'lm_eval/tasks/mmlu/generative/mmlu_astronomy.yaml' 2026-05-11T15:53:39,814 adding 'lm_eval/tasks/mmlu/generative/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,815 adding 'lm_eval/tasks/mmlu/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,816 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_biology.yaml' 2026-05-11T15:53:39,817 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,818 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,820 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,821 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,822 adding 'lm_eval/tasks/mmlu/generative/mmlu_college_physics.yaml' 2026-05-11T15:53:39,823 adding 'lm_eval/tasks/mmlu/generative/mmlu_computer_security.yaml' 2026-05-11T15:53:39,824 adding 'lm_eval/tasks/mmlu/generative/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,825 adding 'lm_eval/tasks/mmlu/generative/mmlu_econometrics.yaml' 2026-05-11T15:53:39,827 adding 'lm_eval/tasks/mmlu/generative/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,828 adding 'lm_eval/tasks/mmlu/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,829 adding 'lm_eval/tasks/mmlu/generative/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,830 adding 'lm_eval/tasks/mmlu/generative/mmlu_global_facts.yaml' 2026-05-11T15:53:39,832 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,833 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,834 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,835 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,836 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,838 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,839 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,840 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,841 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,842 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,844 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,845 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,846 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,848 adding 'lm_eval/tasks/mmlu/generative/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,849 adding 'lm_eval/tasks/mmlu/generative/mmlu_human_aging.yaml' 2026-05-11T15:53:39,850 adding 'lm_eval/tasks/mmlu/generative/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,852 adding 'lm_eval/tasks/mmlu/generative/mmlu_international_law.yaml' 2026-05-11T15:53:39,853 adding 'lm_eval/tasks/mmlu/generative/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,854 adding 'lm_eval/tasks/mmlu/generative/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,855 adding 'lm_eval/tasks/mmlu/generative/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,856 adding 'lm_eval/tasks/mmlu/generative/mmlu_management.yaml' 2026-05-11T15:53:39,858 adding 'lm_eval/tasks/mmlu/generative/mmlu_marketing.yaml' 2026-05-11T15:53:39,859 adding 'lm_eval/tasks/mmlu/generative/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,860 adding 'lm_eval/tasks/mmlu/generative/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,862 adding 'lm_eval/tasks/mmlu/generative/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,863 adding 'lm_eval/tasks/mmlu/generative/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,864 adding 'lm_eval/tasks/mmlu/generative/mmlu_nutrition.yaml' 2026-05-11T15:53:39,865 adding 'lm_eval/tasks/mmlu/generative/mmlu_philosophy.yaml' 2026-05-11T15:53:39,866 adding 'lm_eval/tasks/mmlu/generative/mmlu_prehistory.yaml' 2026-05-11T15:53:39,867 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,869 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_law.yaml' 2026-05-11T15:53:39,870 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,871 adding 'lm_eval/tasks/mmlu/generative/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,873 adding 'lm_eval/tasks/mmlu/generative/mmlu_public_relations.yaml' 2026-05-11T15:53:39,874 adding 'lm_eval/tasks/mmlu/generative/mmlu_security_studies.yaml' 2026-05-11T15:53:39,875 adding 'lm_eval/tasks/mmlu/generative/mmlu_sociology.yaml' 2026-05-11T15:53:39,876 adding 'lm_eval/tasks/mmlu/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,878 adding 'lm_eval/tasks/mmlu/generative/mmlu_virology.yaml' 2026-05-11T15:53:39,879 adding 'lm_eval/tasks/mmlu/generative/mmlu_world_religions.yaml' 2026-05-11T15:53:39,881 adding 'lm_eval/tasks/mmlu-pro-plus/README.md' 2026-05-11T15:53:39,883 adding 'lm_eval/tasks/mmlu-pro-plus/_default_template_yaml' 2026-05-11T15:53:39,884 adding 'lm_eval/tasks/mmlu-pro-plus/_mmlu_pro_plus.yaml' 2026-05-11T15:53:39,885 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_biology.yaml' 2026-05-11T15:53:39,887 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_business.yaml' 2026-05-11T15:53:39,888 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_chemistry.yaml' 2026-05-11T15:53:39,889 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_computer_science.yaml' 2026-05-11T15:53:39,891 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_economics.yaml' 2026-05-11T15:53:39,892 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_engineering.yaml' 2026-05-11T15:53:39,893 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_health.yaml' 2026-05-11T15:53:39,894 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_history.yaml' 2026-05-11T15:53:39,896 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_law.yaml' 2026-05-11T15:53:39,897 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_math.yaml' 2026-05-11T15:53:39,898 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_other.yaml' 2026-05-11T15:53:39,899 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_philosophy.yaml' 2026-05-11T15:53:39,901 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_physics.yaml' 2026-05-11T15:53:39,902 adding 'lm_eval/tasks/mmlu-pro-plus/mmlu_pro_plus_psychology.yaml' 2026-05-11T15:53:39,903 adding 'lm_eval/tasks/mmlu-pro-plus/utils.py' 2026-05-11T15:53:39,907 adding 'lm_eval/tasks/mmlu-redux/generative/README.md' 2026-05-11T15:53:39,908 adding 'lm_eval/tasks/mmlu-redux/generative/_default_template_yaml' 2026-05-11T15:53:39,909 adding 'lm_eval/tasks/mmlu-redux/generative/_mmlu.yaml' 2026-05-11T15:53:39,910 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,911 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_anatomy.yaml' 2026-05-11T15:53:39,912 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_astronomy.yaml' 2026-05-11T15:53:39,914 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,915 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,916 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_biology.yaml' 2026-05-11T15:53:39,917 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,919 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,920 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,921 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,922 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_college_physics.yaml' 2026-05-11T15:53:39,923 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_computer_security.yaml' 2026-05-11T15:53:39,925 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:39,926 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_econometrics.yaml' 2026-05-11T15:53:39,927 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:39,928 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:39,930 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_formal_logic.yaml' 2026-05-11T15:53:39,931 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_global_facts.yaml' 2026-05-11T15:53:39,932 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_biology.yaml' 2026-05-11T15:53:39,933 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:39,934 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:39,936 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:39,937 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_geography.yaml' 2026-05-11T15:53:39,938 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:39,939 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:39,941 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:39,942 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:39,943 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_physics.yaml' 2026-05-11T15:53:39,944 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:39,945 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:39,947 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:39,948 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:39,949 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_human_aging.yaml' 2026-05-11T15:53:39,950 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_human_sexuality.yaml' 2026-05-11T15:53:39,952 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_international_law.yaml' 2026-05-11T15:53:39,953 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_jurisprudence.yaml' 2026-05-11T15:53:39,954 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:39,955 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_machine_learning.yaml' 2026-05-11T15:53:39,956 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_management.yaml' 2026-05-11T15:53:39,958 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_marketing.yaml' 2026-05-11T15:53:39,959 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_medical_genetics.yaml' 2026-05-11T15:53:39,960 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_miscellaneous.yaml' 2026-05-11T15:53:39,961 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_moral_disputes.yaml' 2026-05-11T15:53:39,962 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:39,964 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_nutrition.yaml' 2026-05-11T15:53:39,965 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_philosophy.yaml' 2026-05-11T15:53:39,966 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_prehistory.yaml' 2026-05-11T15:53:39,967 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_accounting.yaml' 2026-05-11T15:53:39,969 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_law.yaml' 2026-05-11T15:53:39,970 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_medicine.yaml' 2026-05-11T15:53:39,971 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_professional_psychology.yaml' 2026-05-11T15:53:39,972 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_public_relations.yaml' 2026-05-11T15:53:39,973 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_security_studies.yaml' 2026-05-11T15:53:39,974 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_sociology.yaml' 2026-05-11T15:53:39,976 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:39,977 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_virology.yaml' 2026-05-11T15:53:39,978 adding 'lm_eval/tasks/mmlu-redux/generative/mmlu_world_religions.yaml' 2026-05-11T15:53:39,980 adding 'lm_eval/tasks/mmlu-redux-spanish/README.md' 2026-05-11T15:53:39,981 adding 'lm_eval/tasks/mmlu-redux-spanish/mmlu-redux-2.0-spanish.yaml' 2026-05-11T15:53:39,984 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/_default_template_spanish_yaml' 2026-05-11T15:53:39,985 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/_mmlu.yaml' 2026-05-11T15:53:39,986 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_abstract_algebra.yaml' 2026-05-11T15:53:39,988 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_anatomy.yaml' 2026-05-11T15:53:39,989 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_astronomy.yaml' 2026-05-11T15:53:39,990 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_business_ethics.yaml' 2026-05-11T15:53:39,991 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_clinical_knowledge.yaml' 2026-05-11T15:53:39,992 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_biology.yaml' 2026-05-11T15:53:39,994 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_chemistry.yaml' 2026-05-11T15:53:39,995 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_computer_science.yaml' 2026-05-11T15:53:39,996 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_mathematics.yaml' 2026-05-11T15:53:39,997 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_medicine.yaml' 2026-05-11T15:53:39,998 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_college_physics.yaml' 2026-05-11T15:53:40,000 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_computer_security.yaml' 2026-05-11T15:53:40,001 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_conceptual_physics.yaml' 2026-05-11T15:53:40,002 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_econometrics.yaml' 2026-05-11T15:53:40,003 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_electrical_engineering.yaml' 2026-05-11T15:53:40,004 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_elementary_mathematics.yaml' 2026-05-11T15:53:40,005 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_formal_logic.yaml' 2026-05-11T15:53:40,006 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_global_facts.yaml' 2026-05-11T15:53:40,007 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_biology.yaml' 2026-05-11T15:53:40,009 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_chemistry.yaml' 2026-05-11T15:53:40,010 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_computer_science.yaml' 2026-05-11T15:53:40,011 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_european_history.yaml' 2026-05-11T15:53:40,012 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_geography.yaml' 2026-05-11T15:53:40,013 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_government_and_politics.yaml' 2026-05-11T15:53:40,015 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_macroeconomics.yaml' 2026-05-11T15:53:40,016 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_mathematics.yaml' 2026-05-11T15:53:40,017 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_microeconomics.yaml' 2026-05-11T15:53:40,018 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_physics.yaml' 2026-05-11T15:53:40,019 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_psychology.yaml' 2026-05-11T15:53:40,021 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_statistics.yaml' 2026-05-11T15:53:40,022 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_us_history.yaml' 2026-05-11T15:53:40,023 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_high_school_world_history.yaml' 2026-05-11T15:53:40,024 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_aging.yaml' 2026-05-11T15:53:40,025 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_human_sexuality.yaml' 2026-05-11T15:53:40,027 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_international_law.yaml' 2026-05-11T15:53:40,028 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_jurisprudence.yaml' 2026-05-11T15:53:40,029 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_logical_fallacies.yaml' 2026-05-11T15:53:40,030 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_machine_learning.yaml' 2026-05-11T15:53:40,031 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_management.yaml' 2026-05-11T15:53:40,033 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_marketing.yaml' 2026-05-11T15:53:40,034 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_medical_genetics.yaml' 2026-05-11T15:53:40,035 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_miscellaneous.yaml' 2026-05-11T15:53:40,036 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_disputes.yaml' 2026-05-11T15:53:40,037 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_moral_scenarios.yaml' 2026-05-11T15:53:40,039 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_nutrition.yaml' 2026-05-11T15:53:40,040 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_philosophy.yaml' 2026-05-11T15:53:40,041 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_prehistory.yaml' 2026-05-11T15:53:40,042 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_accounting.yaml' 2026-05-11T15:53:40,044 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_law.yaml' 2026-05-11T15:53:40,045 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_medicine.yaml' 2026-05-11T15:53:40,046 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_professional_psychology.yaml' 2026-05-11T15:53:40,047 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_public_relations.yaml' 2026-05-11T15:53:40,049 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_security_studies.yaml' 2026-05-11T15:53:40,050 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_sociology.yaml' 2026-05-11T15:53:40,051 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_us_foreign_policy.yaml' 2026-05-11T15:53:40,052 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_virology.yaml' 2026-05-11T15:53:40,053 adding 'lm_eval/tasks/mmlu-redux-spanish/generative/mmlu_world_religions.yaml' 2026-05-11T15:53:40,055 adding 'lm_eval/tasks/mmlu_pro/README.md' 2026-05-11T15:53:40,056 adding 'lm_eval/tasks/mmlu_pro/_default_template_yaml' 2026-05-11T15:53:40,058 adding 'lm_eval/tasks/mmlu_pro/_mmlu_pro.yaml' 2026-05-11T15:53:40,059 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_biology.yaml' 2026-05-11T15:53:40,060 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_business.yaml' 2026-05-11T15:53:40,061 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_chemistry.yaml' 2026-05-11T15:53:40,062 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_computer_science.yaml' 2026-05-11T15:53:40,063 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_economics.yaml' 2026-05-11T15:53:40,065 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_engineering.yaml' 2026-05-11T15:53:40,066 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_health.yaml' 2026-05-11T15:53:40,067 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_history.yaml' 2026-05-11T15:53:40,068 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_law.yaml' 2026-05-11T15:53:40,069 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_math.yaml' 2026-05-11T15:53:40,070 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_other.yaml' 2026-05-11T15:53:40,072 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_philosophy.yaml' 2026-05-11T15:53:40,073 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_physics.yaml' 2026-05-11T15:53:40,074 adding 'lm_eval/tasks/mmlu_pro/mmlu_pro_psychology.yaml' 2026-05-11T15:53:40,075 adding 'lm_eval/tasks/mmlu_pro/utils.py' 2026-05-11T15:53:40,078 adding 'lm_eval/tasks/mmlu_prox/README.md' 2026-05-11T15:53:40,081 adding 'lm_eval/tasks/mmlu_prox/lang_libs.py' 2026-05-11T15:53:40,083 adding 'lm_eval/tasks/mmlu_prox/mmlu_prox_config_generator.py' 2026-05-11T15:53:40,084 adding 'lm_eval/tasks/mmlu_prox/mmlu_prox_lite_config_generator.py' 2026-05-11T15:53:40,087 adding 'lm_eval/tasks/mmlu_prox/af/_af_lite_template_yaml' 2026-05-11T15:53:40,088 adding 'lm_eval/tasks/mmlu_prox/af/_af_template_yaml' 2026-05-11T15:53:40,089 adding 'lm_eval/tasks/mmlu_prox/af/_mmlu_prox_af.yaml' 2026-05-11T15:53:40,090 adding 'lm_eval/tasks/mmlu_prox/af/_mmlu_prox_lite_af.yaml' 2026-05-11T15:53:40,091 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_biology.yaml' 2026-05-11T15:53:40,093 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_business.yaml' 2026-05-11T15:53:40,094 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_chemistry.yaml' 2026-05-11T15:53:40,095 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_computer_science.yaml' 2026-05-11T15:53:40,096 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_economics.yaml' 2026-05-11T15:53:40,097 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_engineering.yaml' 2026-05-11T15:53:40,099 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_health.yaml' 2026-05-11T15:53:40,100 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_history.yaml' 2026-05-11T15:53:40,101 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_law.yaml' 2026-05-11T15:53:40,102 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_math.yaml' 2026-05-11T15:53:40,103 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_other.yaml' 2026-05-11T15:53:40,105 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_philosophy.yaml' 2026-05-11T15:53:40,106 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_physics.yaml' 2026-05-11T15:53:40,107 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_af_psychology.yaml' 2026-05-11T15:53:40,108 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_biology.yaml' 2026-05-11T15:53:40,110 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_business.yaml' 2026-05-11T15:53:40,111 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_chemistry.yaml' 2026-05-11T15:53:40,112 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_computer_science.yaml' 2026-05-11T15:53:40,113 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_economics.yaml' 2026-05-11T15:53:40,114 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_engineering.yaml' 2026-05-11T15:53:40,115 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_health.yaml' 2026-05-11T15:53:40,117 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_history.yaml' 2026-05-11T15:53:40,118 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_law.yaml' 2026-05-11T15:53:40,119 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_math.yaml' 2026-05-11T15:53:40,120 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_other.yaml' 2026-05-11T15:53:40,121 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_philosophy.yaml' 2026-05-11T15:53:40,122 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_physics.yaml' 2026-05-11T15:53:40,123 adding 'lm_eval/tasks/mmlu_prox/af/mmlu_prox_lite_af_psychology.yaml' 2026-05-11T15:53:40,125 adding 'lm_eval/tasks/mmlu_prox/af/utils.py' 2026-05-11T15:53:40,127 adding 'lm_eval/tasks/mmlu_prox/ar/_ar_lite_template_yaml' 2026-05-11T15:53:40,128 adding 'lm_eval/tasks/mmlu_prox/ar/_ar_template_yaml' 2026-05-11T15:53:40,130 adding 'lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_ar.yaml' 2026-05-11T15:53:40,131 adding 'lm_eval/tasks/mmlu_prox/ar/_mmlu_prox_lite_ar.yaml' 2026-05-11T15:53:40,132 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_biology.yaml' 2026-05-11T15:53:40,133 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_business.yaml' 2026-05-11T15:53:40,134 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_chemistry.yaml' 2026-05-11T15:53:40,135 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_computer_science.yaml' 2026-05-11T15:53:40,136 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_economics.yaml' 2026-05-11T15:53:40,138 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_engineering.yaml' 2026-05-11T15:53:40,139 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_health.yaml' 2026-05-11T15:53:40,140 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_history.yaml' 2026-05-11T15:53:40,141 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_law.yaml' 2026-05-11T15:53:40,142 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_math.yaml' 2026-05-11T15:53:40,144 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_other.yaml' 2026-05-11T15:53:40,145 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_philosophy.yaml' 2026-05-11T15:53:40,146 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_physics.yaml' 2026-05-11T15:53:40,147 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_ar_psychology.yaml' 2026-05-11T15:53:40,149 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_biology.yaml' 2026-05-11T15:53:40,150 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_business.yaml' 2026-05-11T15:53:40,151 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_chemistry.yaml' 2026-05-11T15:53:40,152 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_computer_science.yaml' 2026-05-11T15:53:40,153 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_economics.yaml' 2026-05-11T15:53:40,155 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_engineering.yaml' 2026-05-11T15:53:40,156 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_health.yaml' 2026-05-11T15:53:40,157 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_history.yaml' 2026-05-11T15:53:40,158 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_law.yaml' 2026-05-11T15:53:40,159 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_math.yaml' 2026-05-11T15:53:40,161 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_other.yaml' 2026-05-11T15:53:40,162 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_philosophy.yaml' 2026-05-11T15:53:40,163 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_physics.yaml' 2026-05-11T15:53:40,164 adding 'lm_eval/tasks/mmlu_prox/ar/mmlu_prox_lite_ar_psychology.yaml' 2026-05-11T15:53:40,166 adding 'lm_eval/tasks/mmlu_prox/ar/utils.py' 2026-05-11T15:53:40,168 adding 'lm_eval/tasks/mmlu_prox/bn/_bn_lite_template_yaml' 2026-05-11T15:53:40,169 adding 'lm_eval/tasks/mmlu_prox/bn/_bn_template_yaml' 2026-05-11T15:53:40,171 adding 'lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_bn.yaml' 2026-05-11T15:53:40,172 adding 'lm_eval/tasks/mmlu_prox/bn/_mmlu_prox_lite_bn.yaml' 2026-05-11T15:53:40,173 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_biology.yaml' 2026-05-11T15:53:40,174 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_business.yaml' 2026-05-11T15:53:40,175 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_chemistry.yaml' 2026-05-11T15:53:40,176 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_computer_science.yaml' 2026-05-11T15:53:40,177 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_economics.yaml' 2026-05-11T15:53:40,179 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_engineering.yaml' 2026-05-11T15:53:40,180 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_health.yaml' 2026-05-11T15:53:40,181 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_history.yaml' 2026-05-11T15:53:40,182 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_law.yaml' 2026-05-11T15:53:40,183 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_math.yaml' 2026-05-11T15:53:40,185 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_other.yaml' 2026-05-11T15:53:40,186 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_philosophy.yaml' 2026-05-11T15:53:40,187 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_physics.yaml' 2026-05-11T15:53:40,188 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_bn_psychology.yaml' 2026-05-11T15:53:40,189 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_biology.yaml' 2026-05-11T15:53:40,191 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_business.yaml' 2026-05-11T15:53:40,192 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_chemistry.yaml' 2026-05-11T15:53:40,193 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_computer_science.yaml' 2026-05-11T15:53:40,194 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_economics.yaml' 2026-05-11T15:53:40,195 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_engineering.yaml' 2026-05-11T15:53:40,197 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_health.yaml' 2026-05-11T15:53:40,198 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_history.yaml' 2026-05-11T15:53:40,199 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_law.yaml' 2026-05-11T15:53:40,200 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_math.yaml' 2026-05-11T15:53:40,202 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_other.yaml' 2026-05-11T15:53:40,203 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_philosophy.yaml' 2026-05-11T15:53:40,204 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_physics.yaml' 2026-05-11T15:53:40,205 adding 'lm_eval/tasks/mmlu_prox/bn/mmlu_prox_lite_bn_psychology.yaml' 2026-05-11T15:53:40,206 adding 'lm_eval/tasks/mmlu_prox/bn/utils.py' 2026-05-11T15:53:40,209 adding 'lm_eval/tasks/mmlu_prox/cs/_cs_lite_template_yaml' 2026-05-11T15:53:40,210 adding 'lm_eval/tasks/mmlu_prox/cs/_cs_template_yaml' 2026-05-11T15:53:40,211 adding 'lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_cs.yaml' 2026-05-11T15:53:40,213 adding 'lm_eval/tasks/mmlu_prox/cs/_mmlu_prox_lite_cs.yaml' 2026-05-11T15:53:40,214 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_biology.yaml' 2026-05-11T15:53:40,215 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_business.yaml' 2026-05-11T15:53:40,216 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_chemistry.yaml' 2026-05-11T15:53:40,217 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_computer_science.yaml' 2026-05-11T15:53:40,219 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_economics.yaml' 2026-05-11T15:53:40,220 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_engineering.yaml' 2026-05-11T15:53:40,221 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_health.yaml' 2026-05-11T15:53:40,222 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_history.yaml' 2026-05-11T15:53:40,223 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_law.yaml' 2026-05-11T15:53:40,225 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_math.yaml' 2026-05-11T15:53:40,226 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_other.yaml' 2026-05-11T15:53:40,227 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_philosophy.yaml' 2026-05-11T15:53:40,228 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_physics.yaml' 2026-05-11T15:53:40,229 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_cs_psychology.yaml' 2026-05-11T15:53:40,230 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_biology.yaml' 2026-05-11T15:53:40,231 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_business.yaml' 2026-05-11T15:53:40,233 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_chemistry.yaml' 2026-05-11T15:53:40,234 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_computer_science.yaml' 2026-05-11T15:53:40,235 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_economics.yaml' 2026-05-11T15:53:40,236 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_engineering.yaml' 2026-05-11T15:53:40,237 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_health.yaml' 2026-05-11T15:53:40,239 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_history.yaml' 2026-05-11T15:53:40,240 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_law.yaml' 2026-05-11T15:53:40,241 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_math.yaml' 2026-05-11T15:53:40,242 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_other.yaml' 2026-05-11T15:53:40,243 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_philosophy.yaml' 2026-05-11T15:53:40,244 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_physics.yaml' 2026-05-11T15:53:40,246 adding 'lm_eval/tasks/mmlu_prox/cs/mmlu_prox_lite_cs_psychology.yaml' 2026-05-11T15:53:40,247 adding 'lm_eval/tasks/mmlu_prox/cs/utils.py' 2026-05-11T15:53:40,249 adding 'lm_eval/tasks/mmlu_prox/de/_de_lite_template_yaml' 2026-05-11T15:53:40,251 adding 'lm_eval/tasks/mmlu_prox/de/_de_template_yaml' 2026-05-11T15:53:40,252 adding 'lm_eval/tasks/mmlu_prox/de/_mmlu_prox_de.yaml' 2026-05-11T15:53:40,253 adding 'lm_eval/tasks/mmlu_prox/de/_mmlu_prox_lite_de.yaml' 2026-05-11T15:53:40,254 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_biology.yaml' 2026-05-11T15:53:40,255 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_business.yaml' 2026-05-11T15:53:40,256 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_chemistry.yaml' 2026-05-11T15:53:40,258 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_computer_science.yaml' 2026-05-11T15:53:40,259 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_economics.yaml' 2026-05-11T15:53:40,260 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_engineering.yaml' 2026-05-11T15:53:40,261 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_health.yaml' 2026-05-11T15:53:40,263 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_history.yaml' 2026-05-11T15:53:40,264 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_law.yaml' 2026-05-11T15:53:40,265 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_math.yaml' 2026-05-11T15:53:40,266 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_other.yaml' 2026-05-11T15:53:40,267 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_philosophy.yaml' 2026-05-11T15:53:40,269 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_physics.yaml' 2026-05-11T15:53:40,270 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_de_psychology.yaml' 2026-05-11T15:53:40,271 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_biology.yaml' 2026-05-11T15:53:40,272 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_business.yaml' 2026-05-11T15:53:40,274 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_chemistry.yaml' 2026-05-11T15:53:40,275 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_computer_science.yaml' 2026-05-11T15:53:40,276 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_economics.yaml' 2026-05-11T15:53:40,277 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_engineering.yaml' 2026-05-11T15:53:40,278 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_health.yaml' 2026-05-11T15:53:40,280 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_history.yaml' 2026-05-11T15:53:40,281 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_law.yaml' 2026-05-11T15:53:40,282 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_math.yaml' 2026-05-11T15:53:40,283 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_other.yaml' 2026-05-11T15:53:40,284 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_philosophy.yaml' 2026-05-11T15:53:40,285 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_physics.yaml' 2026-05-11T15:53:40,286 adding 'lm_eval/tasks/mmlu_prox/de/mmlu_prox_lite_de_psychology.yaml' 2026-05-11T15:53:40,288 adding 'lm_eval/tasks/mmlu_prox/de/utils.py' 2026-05-11T15:53:40,290 adding 'lm_eval/tasks/mmlu_prox/en/_en_lite_template_yaml' 2026-05-11T15:53:40,291 adding 'lm_eval/tasks/mmlu_prox/en/_en_template_yaml' 2026-05-11T15:53:40,292 adding 'lm_eval/tasks/mmlu_prox/en/_mmlu_prox_en.yaml' 2026-05-11T15:53:40,294 adding 'lm_eval/tasks/mmlu_prox/en/_mmlu_prox_lite_en.yaml' 2026-05-11T15:53:40,295 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_biology.yaml' 2026-05-11T15:53:40,296 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_business.yaml' 2026-05-11T15:53:40,297 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_chemistry.yaml' 2026-05-11T15:53:40,299 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_computer_science.yaml' 2026-05-11T15:53:40,300 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_economics.yaml' 2026-05-11T15:53:40,301 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_engineering.yaml' 2026-05-11T15:53:40,302 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_health.yaml' 2026-05-11T15:53:40,303 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_history.yaml' 2026-05-11T15:53:40,304 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_law.yaml' 2026-05-11T15:53:40,306 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_math.yaml' 2026-05-11T15:53:40,307 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_other.yaml' 2026-05-11T15:53:40,308 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_philosophy.yaml' 2026-05-11T15:53:40,309 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_physics.yaml' 2026-05-11T15:53:40,310 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_en_psychology.yaml' 2026-05-11T15:53:40,312 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_biology.yaml' 2026-05-11T15:53:40,313 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_business.yaml' 2026-05-11T15:53:40,314 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_chemistry.yaml' 2026-05-11T15:53:40,315 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_computer_science.yaml' 2026-05-11T15:53:40,316 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_economics.yaml' 2026-05-11T15:53:40,318 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_engineering.yaml' 2026-05-11T15:53:40,319 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_health.yaml' 2026-05-11T15:53:40,320 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_history.yaml' 2026-05-11T15:53:40,321 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_law.yaml' 2026-05-11T15:53:40,323 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_math.yaml' 2026-05-11T15:53:40,324 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_other.yaml' 2026-05-11T15:53:40,325 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_philosophy.yaml' 2026-05-11T15:53:40,326 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_physics.yaml' 2026-05-11T15:53:40,327 adding 'lm_eval/tasks/mmlu_prox/en/mmlu_prox_lite_en_psychology.yaml' 2026-05-11T15:53:40,329 adding 'lm_eval/tasks/mmlu_prox/en/utils.py' 2026-05-11T15:53:40,331 adding 'lm_eval/tasks/mmlu_prox/es/_es_lite_template_yaml' 2026-05-11T15:53:40,332 adding 'lm_eval/tasks/mmlu_prox/es/_es_template_yaml' 2026-05-11T15:53:40,333 adding 'lm_eval/tasks/mmlu_prox/es/_mmlu_prox_es.yaml' 2026-05-11T15:53:40,335 adding 'lm_eval/tasks/mmlu_prox/es/_mmlu_prox_lite_es.yaml' 2026-05-11T15:53:40,336 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_biology.yaml' 2026-05-11T15:53:40,337 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_business.yaml' 2026-05-11T15:53:40,338 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_chemistry.yaml' 2026-05-11T15:53:40,339 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_computer_science.yaml' 2026-05-11T15:53:40,340 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_economics.yaml' 2026-05-11T15:53:40,341 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_engineering.yaml' 2026-05-11T15:53:40,342 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_health.yaml' 2026-05-11T15:53:40,344 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_history.yaml' 2026-05-11T15:53:40,345 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_law.yaml' 2026-05-11T15:53:40,346 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_math.yaml' 2026-05-11T15:53:40,347 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_other.yaml' 2026-05-11T15:53:40,348 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_philosophy.yaml' 2026-05-11T15:53:40,349 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_physics.yaml' 2026-05-11T15:53:40,350 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_es_psychology.yaml' 2026-05-11T15:53:40,352 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_biology.yaml' 2026-05-11T15:53:40,353 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_business.yaml' 2026-05-11T15:53:40,354 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_chemistry.yaml' 2026-05-11T15:53:40,355 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_computer_science.yaml' 2026-05-11T15:53:40,356 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_economics.yaml' 2026-05-11T15:53:40,357 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_engineering.yaml' 2026-05-11T15:53:40,358 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_health.yaml' 2026-05-11T15:53:40,360 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_history.yaml' 2026-05-11T15:53:40,361 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_law.yaml' 2026-05-11T15:53:40,362 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_math.yaml' 2026-05-11T15:53:40,363 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_other.yaml' 2026-05-11T15:53:40,364 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_philosophy.yaml' 2026-05-11T15:53:40,366 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_physics.yaml' 2026-05-11T15:53:40,367 adding 'lm_eval/tasks/mmlu_prox/es/mmlu_prox_lite_es_psychology.yaml' 2026-05-11T15:53:40,368 adding 'lm_eval/tasks/mmlu_prox/es/utils.py' 2026-05-11T15:53:40,371 adding 'lm_eval/tasks/mmlu_prox/fr/_fr_lite_template_yaml' 2026-05-11T15:53:40,372 adding 'lm_eval/tasks/mmlu_prox/fr/_fr_template_yaml' 2026-05-11T15:53:40,373 adding 'lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_fr.yaml' 2026-05-11T15:53:40,374 adding 'lm_eval/tasks/mmlu_prox/fr/_mmlu_prox_lite_fr.yaml' 2026-05-11T15:53:40,375 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_biology.yaml' 2026-05-11T15:53:40,377 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_business.yaml' 2026-05-11T15:53:40,378 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_chemistry.yaml' 2026-05-11T15:53:40,379 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_computer_science.yaml' 2026-05-11T15:53:40,380 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_economics.yaml' 2026-05-11T15:53:40,382 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_engineering.yaml' 2026-05-11T15:53:40,383 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_health.yaml' 2026-05-11T15:53:40,384 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_history.yaml' 2026-05-11T15:53:40,385 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_law.yaml' 2026-05-11T15:53:40,386 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_math.yaml' 2026-05-11T15:53:40,388 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_other.yaml' 2026-05-11T15:53:40,389 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_philosophy.yaml' 2026-05-11T15:53:40,390 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_physics.yaml' 2026-05-11T15:53:40,391 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_fr_psychology.yaml' 2026-05-11T15:53:40,393 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_biology.yaml' 2026-05-11T15:53:40,394 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_business.yaml' 2026-05-11T15:53:40,395 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_chemistry.yaml' 2026-05-11T15:53:40,396 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_computer_science.yaml' 2026-05-11T15:53:40,397 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_economics.yaml' 2026-05-11T15:53:40,398 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_engineering.yaml' 2026-05-11T15:53:40,399 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_health.yaml' 2026-05-11T15:53:40,400 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_history.yaml' 2026-05-11T15:53:40,401 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_law.yaml' 2026-05-11T15:53:40,402 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_math.yaml' 2026-05-11T15:53:40,404 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_other.yaml' 2026-05-11T15:53:40,405 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_philosophy.yaml' 2026-05-11T15:53:40,406 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_physics.yaml' 2026-05-11T15:53:40,407 adding 'lm_eval/tasks/mmlu_prox/fr/mmlu_prox_lite_fr_psychology.yaml' 2026-05-11T15:53:40,409 adding 'lm_eval/tasks/mmlu_prox/fr/utils.py' 2026-05-11T15:53:40,411 adding 'lm_eval/tasks/mmlu_prox/hi/_hi_lite_template_yaml' 2026-05-11T15:53:40,412 adding 'lm_eval/tasks/mmlu_prox/hi/_hi_template_yaml' 2026-05-11T15:53:40,413 adding 'lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_hi.yaml' 2026-05-11T15:53:40,414 adding 'lm_eval/tasks/mmlu_prox/hi/_mmlu_prox_lite_hi.yaml' 2026-05-11T15:53:40,416 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_biology.yaml' 2026-05-11T15:53:40,417 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_business.yaml' 2026-05-11T15:53:40,418 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_chemistry.yaml' 2026-05-11T15:53:40,419 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_computer_science.yaml' 2026-05-11T15:53:40,421 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_economics.yaml' 2026-05-11T15:53:40,422 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_engineering.yaml' 2026-05-11T15:53:40,423 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_health.yaml' 2026-05-11T15:53:40,424 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_history.yaml' 2026-05-11T15:53:40,426 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_law.yaml' 2026-05-11T15:53:40,427 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_math.yaml' 2026-05-11T15:53:40,428 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_other.yaml' 2026-05-11T15:53:40,429 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_philosophy.yaml' 2026-05-11T15:53:40,430 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_physics.yaml' 2026-05-11T15:53:40,432 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_hi_psychology.yaml' 2026-05-11T15:53:40,433 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_biology.yaml' 2026-05-11T15:53:40,434 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_business.yaml' 2026-05-11T15:53:40,435 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_chemistry.yaml' 2026-05-11T15:53:40,437 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_computer_science.yaml' 2026-05-11T15:53:40,438 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_economics.yaml' 2026-05-11T15:53:40,439 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_engineering.yaml' 2026-05-11T15:53:40,440 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_health.yaml' 2026-05-11T15:53:40,442 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_history.yaml' 2026-05-11T15:53:40,443 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_law.yaml' 2026-05-11T15:53:40,444 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_math.yaml' 2026-05-11T15:53:40,445 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_other.yaml' 2026-05-11T15:53:40,447 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_philosophy.yaml' 2026-05-11T15:53:40,448 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_physics.yaml' 2026-05-11T15:53:40,449 adding 'lm_eval/tasks/mmlu_prox/hi/mmlu_prox_lite_hi_psychology.yaml' 2026-05-11T15:53:40,450 adding 'lm_eval/tasks/mmlu_prox/hi/utils.py' 2026-05-11T15:53:40,453 adding 'lm_eval/tasks/mmlu_prox/hu/_hu_lite_template_yaml' 2026-05-11T15:53:40,454 adding 'lm_eval/tasks/mmlu_prox/hu/_hu_template_yaml' 2026-05-11T15:53:40,455 adding 'lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_hu.yaml' 2026-05-11T15:53:40,456 adding 'lm_eval/tasks/mmlu_prox/hu/_mmlu_prox_lite_hu.yaml' 2026-05-11T15:53:40,457 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_biology.yaml' 2026-05-11T15:53:40,458 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_business.yaml' 2026-05-11T15:53:40,460 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_chemistry.yaml' 2026-05-11T15:53:40,461 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_computer_science.yaml' 2026-05-11T15:53:40,462 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_economics.yaml' 2026-05-11T15:53:40,463 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_engineering.yaml' 2026-05-11T15:53:40,464 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_health.yaml' 2026-05-11T15:53:40,466 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_history.yaml' 2026-05-11T15:53:40,467 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_law.yaml' 2026-05-11T15:53:40,468 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_math.yaml' 2026-05-11T15:53:40,469 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_other.yaml' 2026-05-11T15:53:40,470 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_philosophy.yaml' 2026-05-11T15:53:40,471 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_physics.yaml' 2026-05-11T15:53:40,473 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_hu_psychology.yaml' 2026-05-11T15:53:40,474 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_biology.yaml' 2026-05-11T15:53:40,475 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_business.yaml' 2026-05-11T15:53:40,476 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_chemistry.yaml' 2026-05-11T15:53:40,477 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_computer_science.yaml' 2026-05-11T15:53:40,479 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_economics.yaml' 2026-05-11T15:53:40,480 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_engineering.yaml' 2026-05-11T15:53:40,481 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_health.yaml' 2026-05-11T15:53:40,482 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_history.yaml' 2026-05-11T15:53:40,484 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_law.yaml' 2026-05-11T15:53:40,485 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_math.yaml' 2026-05-11T15:53:40,486 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_other.yaml' 2026-05-11T15:53:40,487 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_philosophy.yaml' 2026-05-11T15:53:40,488 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_physics.yaml' 2026-05-11T15:53:40,490 adding 'lm_eval/tasks/mmlu_prox/hu/mmlu_prox_lite_hu_psychology.yaml' 2026-05-11T15:53:40,491 adding 'lm_eval/tasks/mmlu_prox/hu/utils.py' 2026-05-11T15:53:40,494 adding 'lm_eval/tasks/mmlu_prox/id/_id_lite_template_yaml' 2026-05-11T15:53:40,495 adding 'lm_eval/tasks/mmlu_prox/id/_id_template_yaml' 2026-05-11T15:53:40,496 adding 'lm_eval/tasks/mmlu_prox/id/_mmlu_prox_id.yaml' 2026-05-11T15:53:40,498 adding 'lm_eval/tasks/mmlu_prox/id/_mmlu_prox_lite_id.yaml' 2026-05-11T15:53:40,499 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_biology.yaml' 2026-05-11T15:53:40,500 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_business.yaml' 2026-05-11T15:53:40,501 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_chemistry.yaml' 2026-05-11T15:53:40,502 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_computer_science.yaml' 2026-05-11T15:53:40,504 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_economics.yaml' 2026-05-11T15:53:40,505 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_engineering.yaml' 2026-05-11T15:53:40,506 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_health.yaml' 2026-05-11T15:53:40,507 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_history.yaml' 2026-05-11T15:53:40,508 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_law.yaml' 2026-05-11T15:53:40,509 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_math.yaml' 2026-05-11T15:53:40,510 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_other.yaml' 2026-05-11T15:53:40,511 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_philosophy.yaml' 2026-05-11T15:53:40,513 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_physics.yaml' 2026-05-11T15:53:40,514 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_id_psychology.yaml' 2026-05-11T15:53:40,515 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_biology.yaml' 2026-05-11T15:53:40,516 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_business.yaml' 2026-05-11T15:53:40,517 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_chemistry.yaml' 2026-05-11T15:53:40,518 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_computer_science.yaml' 2026-05-11T15:53:40,520 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_economics.yaml' 2026-05-11T15:53:40,521 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_engineering.yaml' 2026-05-11T15:53:40,522 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_health.yaml' 2026-05-11T15:53:40,523 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_history.yaml' 2026-05-11T15:53:40,524 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_law.yaml' 2026-05-11T15:53:40,525 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_math.yaml' 2026-05-11T15:53:40,526 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_other.yaml' 2026-05-11T15:53:40,528 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_philosophy.yaml' 2026-05-11T15:53:40,529 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_physics.yaml' 2026-05-11T15:53:40,530 adding 'lm_eval/tasks/mmlu_prox/id/mmlu_prox_lite_id_psychology.yaml' 2026-05-11T15:53:40,531 adding 'lm_eval/tasks/mmlu_prox/id/utils.py' 2026-05-11T15:53:40,534 adding 'lm_eval/tasks/mmlu_prox/it/_it_lite_template_yaml' 2026-05-11T15:53:40,535 adding 'lm_eval/tasks/mmlu_prox/it/_it_template_yaml' 2026-05-11T15:53:40,536 adding 'lm_eval/tasks/mmlu_prox/it/_mmlu_prox_it.yaml' 2026-05-11T15:53:40,537 adding 'lm_eval/tasks/mmlu_prox/it/_mmlu_prox_lite_it.yaml' 2026-05-11T15:53:40,539 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_biology.yaml' 2026-05-11T15:53:40,540 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_business.yaml' 2026-05-11T15:53:40,541 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_chemistry.yaml' 2026-05-11T15:53:40,542 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_computer_science.yaml' 2026-05-11T15:53:40,543 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_economics.yaml' 2026-05-11T15:53:40,545 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_engineering.yaml' 2026-05-11T15:53:40,546 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_health.yaml' 2026-05-11T15:53:40,547 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_history.yaml' 2026-05-11T15:53:40,549 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_law.yaml' 2026-05-11T15:53:40,550 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_math.yaml' 2026-05-11T15:53:40,551 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_other.yaml' 2026-05-11T15:53:40,552 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_philosophy.yaml' 2026-05-11T15:53:40,553 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_physics.yaml' 2026-05-11T15:53:40,555 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_it_psychology.yaml' 2026-05-11T15:53:40,556 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_biology.yaml' 2026-05-11T15:53:40,557 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_business.yaml' 2026-05-11T15:53:40,558 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_chemistry.yaml' 2026-05-11T15:53:40,560 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_computer_science.yaml' 2026-05-11T15:53:40,561 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_economics.yaml' 2026-05-11T15:53:40,562 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_engineering.yaml' 2026-05-11T15:53:40,563 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_health.yaml' 2026-05-11T15:53:40,564 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_history.yaml' 2026-05-11T15:53:40,565 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_law.yaml' 2026-05-11T15:53:40,566 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_math.yaml' 2026-05-11T15:53:40,568 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_other.yaml' 2026-05-11T15:53:40,569 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_philosophy.yaml' 2026-05-11T15:53:40,570 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_physics.yaml' 2026-05-11T15:53:40,571 adding 'lm_eval/tasks/mmlu_prox/it/mmlu_prox_lite_it_psychology.yaml' 2026-05-11T15:53:40,572 adding 'lm_eval/tasks/mmlu_prox/it/utils.py' 2026-05-11T15:53:40,575 adding 'lm_eval/tasks/mmlu_prox/ja/_ja_lite_template_yaml' 2026-05-11T15:53:40,576 adding 'lm_eval/tasks/mmlu_prox/ja/_ja_template_yaml' 2026-05-11T15:53:40,577 adding 'lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_ja.yaml' 2026-05-11T15:53:40,578 adding 'lm_eval/tasks/mmlu_prox/ja/_mmlu_prox_lite_ja.yaml' 2026-05-11T15:53:40,579 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_biology.yaml' 2026-05-11T15:53:40,581 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_business.yaml' 2026-05-11T15:53:40,582 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_chemistry.yaml' 2026-05-11T15:53:40,583 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_computer_science.yaml' 2026-05-11T15:53:40,584 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_economics.yaml' 2026-05-11T15:53:40,585 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_engineering.yaml' 2026-05-11T15:53:40,586 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_health.yaml' 2026-05-11T15:53:40,588 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_history.yaml' 2026-05-11T15:53:40,589 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_law.yaml' 2026-05-11T15:53:40,590 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_math.yaml' 2026-05-11T15:53:40,591 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_other.yaml' 2026-05-11T15:53:40,593 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_philosophy.yaml' 2026-05-11T15:53:40,594 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_physics.yaml' 2026-05-11T15:53:40,595 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_ja_psychology.yaml' 2026-05-11T15:53:40,597 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_biology.yaml' 2026-05-11T15:53:40,598 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_business.yaml' 2026-05-11T15:53:40,599 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_chemistry.yaml' 2026-05-11T15:53:40,600 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_computer_science.yaml' 2026-05-11T15:53:40,602 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_economics.yaml' 2026-05-11T15:53:40,603 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_engineering.yaml' 2026-05-11T15:53:40,604 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_health.yaml' 2026-05-11T15:53:40,605 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_history.yaml' 2026-05-11T15:53:40,607 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_law.yaml' 2026-05-11T15:53:40,608 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_math.yaml' 2026-05-11T15:53:40,609 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_other.yaml' 2026-05-11T15:53:40,610 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_philosophy.yaml' 2026-05-11T15:53:40,612 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_physics.yaml' 2026-05-11T15:53:40,613 adding 'lm_eval/tasks/mmlu_prox/ja/mmlu_prox_lite_ja_psychology.yaml' 2026-05-11T15:53:40,614 adding 'lm_eval/tasks/mmlu_prox/ja/utils.py' 2026-05-11T15:53:40,617 adding 'lm_eval/tasks/mmlu_prox/ko/_ko_lite_template_yaml' 2026-05-11T15:53:40,618 adding 'lm_eval/tasks/mmlu_prox/ko/_ko_template_yaml' 2026-05-11T15:53:40,619 adding 'lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_ko.yaml' 2026-05-11T15:53:40,620 adding 'lm_eval/tasks/mmlu_prox/ko/_mmlu_prox_lite_ko.yaml' 2026-05-11T15:53:40,622 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_biology.yaml' 2026-05-11T15:53:40,623 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_business.yaml' 2026-05-11T15:53:40,624 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_chemistry.yaml' 2026-05-11T15:53:40,625 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_computer_science.yaml' 2026-05-11T15:53:40,626 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_economics.yaml' 2026-05-11T15:53:40,627 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_engineering.yaml' 2026-05-11T15:53:40,628 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_health.yaml' 2026-05-11T15:53:40,630 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_history.yaml' 2026-05-11T15:53:40,631 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_law.yaml' 2026-05-11T15:53:40,632 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_math.yaml' 2026-05-11T15:53:40,633 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_other.yaml' 2026-05-11T15:53:40,634 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_philosophy.yaml' 2026-05-11T15:53:40,636 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_physics.yaml' 2026-05-11T15:53:40,637 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_ko_psychology.yaml' 2026-05-11T15:53:40,638 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_biology.yaml' 2026-05-11T15:53:40,639 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_business.yaml' 2026-05-11T15:53:40,640 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_chemistry.yaml' 2026-05-11T15:53:40,641 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_computer_science.yaml' 2026-05-11T15:53:40,643 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_economics.yaml' 2026-05-11T15:53:40,644 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_engineering.yaml' 2026-05-11T15:53:40,645 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_health.yaml' 2026-05-11T15:53:40,646 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_history.yaml' 2026-05-11T15:53:40,647 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_law.yaml' 2026-05-11T15:53:40,649 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_math.yaml' 2026-05-11T15:53:40,650 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_other.yaml' 2026-05-11T15:53:40,651 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_philosophy.yaml' 2026-05-11T15:53:40,652 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_physics.yaml' 2026-05-11T15:53:40,653 adding 'lm_eval/tasks/mmlu_prox/ko/mmlu_prox_lite_ko_psychology.yaml' 2026-05-11T15:53:40,655 adding 'lm_eval/tasks/mmlu_prox/ko/utils.py' 2026-05-11T15:53:40,657 adding 'lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_lite_mr.yaml' 2026-05-11T15:53:40,658 adding 'lm_eval/tasks/mmlu_prox/mr/_mmlu_prox_mr.yaml' 2026-05-11T15:53:40,660 adding 'lm_eval/tasks/mmlu_prox/mr/_mr_lite_template_yaml' 2026-05-11T15:53:40,661 adding 'lm_eval/tasks/mmlu_prox/mr/_mr_template_yaml' 2026-05-11T15:53:40,662 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_biology.yaml' 2026-05-11T15:53:40,663 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_business.yaml' 2026-05-11T15:53:40,665 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_chemistry.yaml' 2026-05-11T15:53:40,666 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_computer_science.yaml' 2026-05-11T15:53:40,667 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_economics.yaml' 2026-05-11T15:53:40,668 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_engineering.yaml' 2026-05-11T15:53:40,670 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_health.yaml' 2026-05-11T15:53:40,671 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_history.yaml' 2026-05-11T15:53:40,672 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_law.yaml' 2026-05-11T15:53:40,673 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_math.yaml' 2026-05-11T15:53:40,674 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_other.yaml' 2026-05-11T15:53:40,676 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_philosophy.yaml' 2026-05-11T15:53:40,677 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_physics.yaml' 2026-05-11T15:53:40,678 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_lite_mr_psychology.yaml' 2026-05-11T15:53:40,679 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_biology.yaml' 2026-05-11T15:53:40,680 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_business.yaml' 2026-05-11T15:53:40,681 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_chemistry.yaml' 2026-05-11T15:53:40,682 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_computer_science.yaml' 2026-05-11T15:53:40,683 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_economics.yaml' 2026-05-11T15:53:40,685 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_engineering.yaml' 2026-05-11T15:53:40,686 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_health.yaml' 2026-05-11T15:53:40,687 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_history.yaml' 2026-05-11T15:53:40,688 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_law.yaml' 2026-05-11T15:53:40,690 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_math.yaml' 2026-05-11T15:53:40,691 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_other.yaml' 2026-05-11T15:53:40,692 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_philosophy.yaml' 2026-05-11T15:53:40,693 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_physics.yaml' 2026-05-11T15:53:40,694 adding 'lm_eval/tasks/mmlu_prox/mr/mmlu_prox_mr_psychology.yaml' 2026-05-11T15:53:40,696 adding 'lm_eval/tasks/mmlu_prox/mr/utils.py' 2026-05-11T15:53:40,698 adding 'lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_lite_ne.yaml' 2026-05-11T15:53:40,699 adding 'lm_eval/tasks/mmlu_prox/ne/_mmlu_prox_ne.yaml' 2026-05-11T15:53:40,700 adding 'lm_eval/tasks/mmlu_prox/ne/_ne_lite_template_yaml' 2026-05-11T15:53:40,702 adding 'lm_eval/tasks/mmlu_prox/ne/_ne_template_yaml' 2026-05-11T15:53:40,703 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_biology.yaml' 2026-05-11T15:53:40,704 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_business.yaml' 2026-05-11T15:53:40,705 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_chemistry.yaml' 2026-05-11T15:53:40,707 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_computer_science.yaml' 2026-05-11T15:53:40,708 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_economics.yaml' 2026-05-11T15:53:40,709 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_engineering.yaml' 2026-05-11T15:53:40,710 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_health.yaml' 2026-05-11T15:53:40,712 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_history.yaml' 2026-05-11T15:53:40,713 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_law.yaml' 2026-05-11T15:53:40,714 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_math.yaml' 2026-05-11T15:53:40,715 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_other.yaml' 2026-05-11T15:53:40,716 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_philosophy.yaml' 2026-05-11T15:53:40,718 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_physics.yaml' 2026-05-11T15:53:40,719 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_lite_ne_psychology.yaml' 2026-05-11T15:53:40,720 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_biology.yaml' 2026-05-11T15:53:40,721 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_business.yaml' 2026-05-11T15:53:40,723 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_chemistry.yaml' 2026-05-11T15:53:40,724 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_computer_science.yaml' 2026-05-11T15:53:40,725 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_economics.yaml' 2026-05-11T15:53:40,726 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_engineering.yaml' 2026-05-11T15:53:40,728 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_health.yaml' 2026-05-11T15:53:40,729 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_history.yaml' 2026-05-11T15:53:40,730 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_law.yaml' 2026-05-11T15:53:40,731 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_math.yaml' 2026-05-11T15:53:40,733 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_other.yaml' 2026-05-11T15:53:40,734 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_philosophy.yaml' 2026-05-11T15:53:40,735 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_physics.yaml' 2026-05-11T15:53:40,736 adding 'lm_eval/tasks/mmlu_prox/ne/mmlu_prox_ne_psychology.yaml' 2026-05-11T15:53:40,737 adding 'lm_eval/tasks/mmlu_prox/ne/utils.py' 2026-05-11T15:53:40,740 adding 'lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_lite_pt.yaml' 2026-05-11T15:53:40,741 adding 'lm_eval/tasks/mmlu_prox/pt/_mmlu_prox_pt.yaml' 2026-05-11T15:53:40,742 adding 'lm_eval/tasks/mmlu_prox/pt/_pt_lite_template_yaml' 2026-05-11T15:53:40,743 adding 'lm_eval/tasks/mmlu_prox/pt/_pt_template_yaml' 2026-05-11T15:53:40,745 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_biology.yaml' 2026-05-11T15:53:40,746 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_business.yaml' 2026-05-11T15:53:40,747 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_chemistry.yaml' 2026-05-11T15:53:40,749 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_computer_science.yaml' 2026-05-11T15:53:40,750 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_economics.yaml' 2026-05-11T15:53:40,751 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_engineering.yaml' 2026-05-11T15:53:40,752 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_health.yaml' 2026-05-11T15:53:40,753 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_history.yaml' 2026-05-11T15:53:40,755 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_law.yaml' 2026-05-11T15:53:40,756 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_math.yaml' 2026-05-11T15:53:40,757 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_other.yaml' 2026-05-11T15:53:40,758 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_philosophy.yaml' 2026-05-11T15:53:40,759 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_physics.yaml' 2026-05-11T15:53:40,761 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_lite_pt_psychology.yaml' 2026-05-11T15:53:40,762 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_biology.yaml' 2026-05-11T15:53:40,763 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_business.yaml' 2026-05-11T15:53:40,764 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_chemistry.yaml' 2026-05-11T15:53:40,765 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_computer_science.yaml' 2026-05-11T15:53:40,767 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_economics.yaml' 2026-05-11T15:53:40,768 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_engineering.yaml' 2026-05-11T15:53:40,769 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_health.yaml' 2026-05-11T15:53:40,770 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_history.yaml' 2026-05-11T15:53:40,772 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_law.yaml' 2026-05-11T15:53:40,773 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_math.yaml' 2026-05-11T15:53:40,774 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_other.yaml' 2026-05-11T15:53:40,775 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_philosophy.yaml' 2026-05-11T15:53:40,777 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_physics.yaml' 2026-05-11T15:53:40,778 adding 'lm_eval/tasks/mmlu_prox/pt/mmlu_prox_pt_psychology.yaml' 2026-05-11T15:53:40,779 adding 'lm_eval/tasks/mmlu_prox/pt/utils.py' 2026-05-11T15:53:40,782 adding 'lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_lite_ru.yaml' 2026-05-11T15:53:40,783 adding 'lm_eval/tasks/mmlu_prox/ru/_mmlu_prox_ru.yaml' 2026-05-11T15:53:40,784 adding 'lm_eval/tasks/mmlu_prox/ru/_ru_lite_template_yaml' 2026-05-11T15:53:40,785 adding 'lm_eval/tasks/mmlu_prox/ru/_ru_template_yaml' 2026-05-11T15:53:40,787 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_biology.yaml' 2026-05-11T15:53:40,788 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_business.yaml' 2026-05-11T15:53:40,789 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_chemistry.yaml' 2026-05-11T15:53:40,791 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_computer_science.yaml' 2026-05-11T15:53:40,792 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_economics.yaml' 2026-05-11T15:53:40,793 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_engineering.yaml' 2026-05-11T15:53:40,794 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_health.yaml' 2026-05-11T15:53:40,795 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_history.yaml' 2026-05-11T15:53:40,796 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_law.yaml' 2026-05-11T15:53:40,797 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_math.yaml' 2026-05-11T15:53:40,799 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_other.yaml' 2026-05-11T15:53:40,800 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_philosophy.yaml' 2026-05-11T15:53:40,801 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_physics.yaml' 2026-05-11T15:53:40,802 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_lite_ru_psychology.yaml' 2026-05-11T15:53:40,803 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_biology.yaml' 2026-05-11T15:53:40,804 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_business.yaml' 2026-05-11T15:53:40,806 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_chemistry.yaml' 2026-05-11T15:53:40,807 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_computer_science.yaml' 2026-05-11T15:53:40,808 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_economics.yaml' 2026-05-11T15:53:40,809 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_engineering.yaml' 2026-05-11T15:53:40,810 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_health.yaml' 2026-05-11T15:53:40,812 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_history.yaml' 2026-05-11T15:53:40,813 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_law.yaml' 2026-05-11T15:53:40,814 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_math.yaml' 2026-05-11T15:53:40,815 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_other.yaml' 2026-05-11T15:53:40,817 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_philosophy.yaml' 2026-05-11T15:53:40,818 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_physics.yaml' 2026-05-11T15:53:40,819 adding 'lm_eval/tasks/mmlu_prox/ru/mmlu_prox_ru_psychology.yaml' 2026-05-11T15:53:40,820 adding 'lm_eval/tasks/mmlu_prox/ru/utils.py' 2026-05-11T15:53:40,823 adding 'lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_lite_sr.yaml' 2026-05-11T15:53:40,824 adding 'lm_eval/tasks/mmlu_prox/sr/_mmlu_prox_sr.yaml' 2026-05-11T15:53:40,825 adding 'lm_eval/tasks/mmlu_prox/sr/_sr_lite_template_yaml' 2026-05-11T15:53:40,827 adding 'lm_eval/tasks/mmlu_prox/sr/_sr_template_yaml' 2026-05-11T15:53:40,828 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_biology.yaml' 2026-05-11T15:53:40,829 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_business.yaml' 2026-05-11T15:53:40,831 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_chemistry.yaml' 2026-05-11T15:53:40,832 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_computer_science.yaml' 2026-05-11T15:53:40,833 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_economics.yaml' 2026-05-11T15:53:40,834 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_engineering.yaml' 2026-05-11T15:53:40,835 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_health.yaml' 2026-05-11T15:53:40,837 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_history.yaml' 2026-05-11T15:53:40,838 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_law.yaml' 2026-05-11T15:53:40,839 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_math.yaml' 2026-05-11T15:53:40,840 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_other.yaml' 2026-05-11T15:53:40,842 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_philosophy.yaml' 2026-05-11T15:53:40,843 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_physics.yaml' 2026-05-11T15:53:40,844 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_lite_sr_psychology.yaml' 2026-05-11T15:53:40,845 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_biology.yaml' 2026-05-11T15:53:40,847 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_business.yaml' 2026-05-11T15:53:40,848 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_chemistry.yaml' 2026-05-11T15:53:40,849 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_computer_science.yaml' 2026-05-11T15:53:40,850 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_economics.yaml' 2026-05-11T15:53:40,851 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_engineering.yaml' 2026-05-11T15:53:40,852 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_health.yaml' 2026-05-11T15:53:40,854 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_history.yaml' 2026-05-11T15:53:40,855 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_law.yaml' 2026-05-11T15:53:40,856 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_math.yaml' 2026-05-11T15:53:40,857 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_other.yaml' 2026-05-11T15:53:40,858 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_philosophy.yaml' 2026-05-11T15:53:40,859 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_physics.yaml' 2026-05-11T15:53:40,861 adding 'lm_eval/tasks/mmlu_prox/sr/mmlu_prox_sr_psychology.yaml' 2026-05-11T15:53:40,862 adding 'lm_eval/tasks/mmlu_prox/sr/utils.py' 2026-05-11T15:53:40,864 adding 'lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_lite_sw.yaml' 2026-05-11T15:53:40,865 adding 'lm_eval/tasks/mmlu_prox/sw/_mmlu_prox_sw.yaml' 2026-05-11T15:53:40,866 adding 'lm_eval/tasks/mmlu_prox/sw/_sw_lite_template_yaml' 2026-05-11T15:53:40,868 adding 'lm_eval/tasks/mmlu_prox/sw/_sw_template_yaml' 2026-05-11T15:53:40,869 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_biology.yaml' 2026-05-11T15:53:40,870 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_business.yaml' 2026-05-11T15:53:40,871 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_chemistry.yaml' 2026-05-11T15:53:40,872 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_computer_science.yaml' 2026-05-11T15:53:40,874 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_economics.yaml' 2026-05-11T15:53:40,875 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_engineering.yaml' 2026-05-11T15:53:40,876 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_health.yaml' 2026-05-11T15:53:40,877 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_history.yaml' 2026-05-11T15:53:40,879 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_law.yaml' 2026-05-11T15:53:40,880 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_math.yaml' 2026-05-11T15:53:40,881 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_other.yaml' 2026-05-11T15:53:40,882 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_philosophy.yaml' 2026-05-11T15:53:40,883 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_physics.yaml' 2026-05-11T15:53:40,885 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_lite_sw_psychology.yaml' 2026-05-11T15:53:40,886 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_biology.yaml' 2026-05-11T15:53:40,887 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_business.yaml' 2026-05-11T15:53:40,888 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_chemistry.yaml' 2026-05-11T15:53:40,889 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_computer_science.yaml' 2026-05-11T15:53:40,891 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_economics.yaml' 2026-05-11T15:53:40,892 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_engineering.yaml' 2026-05-11T15:53:40,893 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_health.yaml' 2026-05-11T15:53:40,894 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_history.yaml' 2026-05-11T15:53:40,896 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_law.yaml' 2026-05-11T15:53:40,897 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_math.yaml' 2026-05-11T15:53:40,898 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_other.yaml' 2026-05-11T15:53:40,899 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_philosophy.yaml' 2026-05-11T15:53:40,900 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_physics.yaml' 2026-05-11T15:53:40,902 adding 'lm_eval/tasks/mmlu_prox/sw/mmlu_prox_sw_psychology.yaml' 2026-05-11T15:53:40,903 adding 'lm_eval/tasks/mmlu_prox/sw/utils.py' 2026-05-11T15:53:40,905 adding 'lm_eval/tasks/mmlu_prox/te/_mmlu_prox_lite_te.yaml' 2026-05-11T15:53:40,906 adding 'lm_eval/tasks/mmlu_prox/te/_mmlu_prox_te.yaml' 2026-05-11T15:53:40,908 adding 'lm_eval/tasks/mmlu_prox/te/_te_lite_template_yaml' 2026-05-11T15:53:40,909 adding 'lm_eval/tasks/mmlu_prox/te/_te_template_yaml' 2026-05-11T15:53:40,910 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_biology.yaml' 2026-05-11T15:53:40,912 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_business.yaml' 2026-05-11T15:53:40,913 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_chemistry.yaml' 2026-05-11T15:53:40,914 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_computer_science.yaml' 2026-05-11T15:53:40,915 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_economics.yaml' 2026-05-11T15:53:40,917 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_engineering.yaml' 2026-05-11T15:53:40,918 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_health.yaml' 2026-05-11T15:53:40,919 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_history.yaml' 2026-05-11T15:53:40,921 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_law.yaml' 2026-05-11T15:53:40,922 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_math.yaml' 2026-05-11T15:53:40,923 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_other.yaml' 2026-05-11T15:53:40,924 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_philosophy.yaml' 2026-05-11T15:53:40,926 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_physics.yaml' 2026-05-11T15:53:40,927 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_lite_te_psychology.yaml' 2026-05-11T15:53:40,928 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_biology.yaml' 2026-05-11T15:53:40,930 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_business.yaml' 2026-05-11T15:53:40,931 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_chemistry.yaml' 2026-05-11T15:53:40,932 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_computer_science.yaml' 2026-05-11T15:53:40,933 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_economics.yaml' 2026-05-11T15:53:40,935 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_engineering.yaml' 2026-05-11T15:53:40,936 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_health.yaml' 2026-05-11T15:53:40,937 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_history.yaml' 2026-05-11T15:53:40,939 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_law.yaml' 2026-05-11T15:53:40,940 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_math.yaml' 2026-05-11T15:53:40,941 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_other.yaml' 2026-05-11T15:53:40,943 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_philosophy.yaml' 2026-05-11T15:53:40,944 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_physics.yaml' 2026-05-11T15:53:40,945 adding 'lm_eval/tasks/mmlu_prox/te/mmlu_prox_te_psychology.yaml' 2026-05-11T15:53:40,947 adding 'lm_eval/tasks/mmlu_prox/te/utils.py' 2026-05-11T15:53:40,949 adding 'lm_eval/tasks/mmlu_prox/template/_lang_template_yaml' 2026-05-11T15:53:40,950 adding 'lm_eval/tasks/mmlu_prox/template/utils.py' 2026-05-11T15:53:40,952 adding 'lm_eval/tasks/mmlu_prox/th/_mmlu_prox_lite_th.yaml' 2026-05-11T15:53:40,953 adding 'lm_eval/tasks/mmlu_prox/th/_mmlu_prox_th.yaml' 2026-05-11T15:53:40,955 adding 'lm_eval/tasks/mmlu_prox/th/_th_lite_template_yaml' 2026-05-11T15:53:40,956 adding 'lm_eval/tasks/mmlu_prox/th/_th_template_yaml' 2026-05-11T15:53:40,958 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_biology.yaml' 2026-05-11T15:53:40,959 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_business.yaml' 2026-05-11T15:53:40,960 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_chemistry.yaml' 2026-05-11T15:53:40,962 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_computer_science.yaml' 2026-05-11T15:53:40,963 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_economics.yaml' 2026-05-11T15:53:40,964 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_engineering.yaml' 2026-05-11T15:53:40,965 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_health.yaml' 2026-05-11T15:53:40,967 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_history.yaml' 2026-05-11T15:53:40,968 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_law.yaml' 2026-05-11T15:53:40,969 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_math.yaml' 2026-05-11T15:53:40,970 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_other.yaml' 2026-05-11T15:53:40,971 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_philosophy.yaml' 2026-05-11T15:53:40,972 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_physics.yaml' 2026-05-11T15:53:40,974 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_lite_th_psychology.yaml' 2026-05-11T15:53:40,975 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_biology.yaml' 2026-05-11T15:53:40,976 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_business.yaml' 2026-05-11T15:53:40,978 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_chemistry.yaml' 2026-05-11T15:53:40,979 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_computer_science.yaml' 2026-05-11T15:53:40,980 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_economics.yaml' 2026-05-11T15:53:40,982 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_engineering.yaml' 2026-05-11T15:53:40,983 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_health.yaml' 2026-05-11T15:53:40,984 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_history.yaml' 2026-05-11T15:53:40,985 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_law.yaml' 2026-05-11T15:53:40,987 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_math.yaml' 2026-05-11T15:53:40,988 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_other.yaml' 2026-05-11T15:53:40,989 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_philosophy.yaml' 2026-05-11T15:53:40,990 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_physics.yaml' 2026-05-11T15:53:40,992 adding 'lm_eval/tasks/mmlu_prox/th/mmlu_prox_th_psychology.yaml' 2026-05-11T15:53:40,993 adding 'lm_eval/tasks/mmlu_prox/th/utils.py' 2026-05-11T15:53:40,995 adding 'lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_lite_uk.yaml' 2026-05-11T15:53:40,996 adding 'lm_eval/tasks/mmlu_prox/uk/_mmlu_prox_uk.yaml' 2026-05-11T15:53:40,998 adding 'lm_eval/tasks/mmlu_prox/uk/_uk_lite_template_yaml' 2026-05-11T15:53:40,999 adding 'lm_eval/tasks/mmlu_prox/uk/_uk_template_yaml' 2026-05-11T15:53:41,000 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_biology.yaml' 2026-05-11T15:53:41,001 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_business.yaml' 2026-05-11T15:53:41,003 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_chemistry.yaml' 2026-05-11T15:53:41,004 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_computer_science.yaml' 2026-05-11T15:53:41,005 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_economics.yaml' 2026-05-11T15:53:41,006 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_engineering.yaml' 2026-05-11T15:53:41,008 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_health.yaml' 2026-05-11T15:53:41,009 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_history.yaml' 2026-05-11T15:53:41,010 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_law.yaml' 2026-05-11T15:53:41,011 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_math.yaml' 2026-05-11T15:53:41,012 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_other.yaml' 2026-05-11T15:53:41,014 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_philosophy.yaml' 2026-05-11T15:53:41,015 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_physics.yaml' 2026-05-11T15:53:41,016 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_lite_uk_psychology.yaml' 2026-05-11T15:53:41,017 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_biology.yaml' 2026-05-11T15:53:41,019 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_business.yaml' 2026-05-11T15:53:41,020 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_chemistry.yaml' 2026-05-11T15:53:41,021 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_computer_science.yaml' 2026-05-11T15:53:41,022 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_economics.yaml' 2026-05-11T15:53:41,023 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_engineering.yaml' 2026-05-11T15:53:41,025 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_health.yaml' 2026-05-11T15:53:41,026 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_history.yaml' 2026-05-11T15:53:41,027 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_law.yaml' 2026-05-11T15:53:41,030 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_math.yaml' 2026-05-11T15:53:41,031 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_other.yaml' 2026-05-11T15:53:41,032 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_philosophy.yaml' 2026-05-11T15:53:41,033 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_physics.yaml' 2026-05-11T15:53:41,034 adding 'lm_eval/tasks/mmlu_prox/uk/mmlu_prox_uk_psychology.yaml' 2026-05-11T15:53:41,035 adding 'lm_eval/tasks/mmlu_prox/uk/utils.py' 2026-05-11T15:53:41,038 adding 'lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_lite_ur.yaml' 2026-05-11T15:53:41,039 adding 'lm_eval/tasks/mmlu_prox/ur/_mmlu_prox_ur.yaml' 2026-05-11T15:53:41,040 adding 'lm_eval/tasks/mmlu_prox/ur/_ur_lite_template_yaml' 2026-05-11T15:53:41,042 adding 'lm_eval/tasks/mmlu_prox/ur/_ur_template_yaml' 2026-05-11T15:53:41,043 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_biology.yaml' 2026-05-11T15:53:41,044 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_business.yaml' 2026-05-11T15:53:41,045 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_chemistry.yaml' 2026-05-11T15:53:41,046 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_computer_science.yaml' 2026-05-11T15:53:41,047 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_economics.yaml' 2026-05-11T15:53:41,049 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_engineering.yaml' 2026-05-11T15:53:41,050 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_health.yaml' 2026-05-11T15:53:41,051 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_history.yaml' 2026-05-11T15:53:41,052 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_law.yaml' 2026-05-11T15:53:41,053 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_math.yaml' 2026-05-11T15:53:41,054 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_other.yaml' 2026-05-11T15:53:41,056 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_philosophy.yaml' 2026-05-11T15:53:41,057 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_physics.yaml' 2026-05-11T15:53:41,058 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_lite_ur_psychology.yaml' 2026-05-11T15:53:41,059 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_biology.yaml' 2026-05-11T15:53:41,060 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_business.yaml' 2026-05-11T15:53:41,062 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_chemistry.yaml' 2026-05-11T15:53:41,063 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_computer_science.yaml' 2026-05-11T15:53:41,064 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_economics.yaml' 2026-05-11T15:53:41,065 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_engineering.yaml' 2026-05-11T15:53:41,067 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_health.yaml' 2026-05-11T15:53:41,068 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_history.yaml' 2026-05-11T15:53:41,069 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_law.yaml' 2026-05-11T15:53:41,070 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_math.yaml' 2026-05-11T15:53:41,072 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_other.yaml' 2026-05-11T15:53:41,073 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_philosophy.yaml' 2026-05-11T15:53:41,074 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_physics.yaml' 2026-05-11T15:53:41,075 adding 'lm_eval/tasks/mmlu_prox/ur/mmlu_prox_ur_psychology.yaml' 2026-05-11T15:53:41,076 adding 'lm_eval/tasks/mmlu_prox/ur/utils.py' 2026-05-11T15:53:41,079 adding 'lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_lite_vi.yaml' 2026-05-11T15:53:41,080 adding 'lm_eval/tasks/mmlu_prox/vi/_mmlu_prox_vi.yaml' 2026-05-11T15:53:41,081 adding 'lm_eval/tasks/mmlu_prox/vi/_vi_lite_template_yaml' 2026-05-11T15:53:41,082 adding 'lm_eval/tasks/mmlu_prox/vi/_vi_template_yaml' 2026-05-11T15:53:41,083 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_biology.yaml' 2026-05-11T15:53:41,085 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_business.yaml' 2026-05-11T15:53:41,086 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_chemistry.yaml' 2026-05-11T15:53:41,087 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_computer_science.yaml' 2026-05-11T15:53:41,088 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_economics.yaml' 2026-05-11T15:53:41,089 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_engineering.yaml' 2026-05-11T15:53:41,090 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_health.yaml' 2026-05-11T15:53:41,091 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_history.yaml' 2026-05-11T15:53:41,092 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_law.yaml' 2026-05-11T15:53:41,094 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_math.yaml' 2026-05-11T15:53:41,095 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_other.yaml' 2026-05-11T15:53:41,096 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_philosophy.yaml' 2026-05-11T15:53:41,097 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_physics.yaml' 2026-05-11T15:53:41,099 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_lite_vi_psychology.yaml' 2026-05-11T15:53:41,100 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_biology.yaml' 2026-05-11T15:53:41,101 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_business.yaml' 2026-05-11T15:53:41,102 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_chemistry.yaml' 2026-05-11T15:53:41,103 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_computer_science.yaml' 2026-05-11T15:53:41,105 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_economics.yaml' 2026-05-11T15:53:41,106 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_engineering.yaml' 2026-05-11T15:53:41,107 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_health.yaml' 2026-05-11T15:53:41,108 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_history.yaml' 2026-05-11T15:53:41,109 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_law.yaml' 2026-05-11T15:53:41,110 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_math.yaml' 2026-05-11T15:53:41,112 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_other.yaml' 2026-05-11T15:53:41,113 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_philosophy.yaml' 2026-05-11T15:53:41,114 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_physics.yaml' 2026-05-11T15:53:41,116 adding 'lm_eval/tasks/mmlu_prox/vi/mmlu_prox_vi_psychology.yaml' 2026-05-11T15:53:41,117 adding 'lm_eval/tasks/mmlu_prox/vi/utils.py' 2026-05-11T15:53:41,119 adding 'lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_lite_wo.yaml' 2026-05-11T15:53:41,121 adding 'lm_eval/tasks/mmlu_prox/wo/_mmlu_prox_wo.yaml' 2026-05-11T15:53:41,122 adding 'lm_eval/tasks/mmlu_prox/wo/_wo_lite_template_yaml' 2026-05-11T15:53:41,123 adding 'lm_eval/tasks/mmlu_prox/wo/_wo_template_yaml' 2026-05-11T15:53:41,124 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_biology.yaml' 2026-05-11T15:53:41,125 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_business.yaml' 2026-05-11T15:53:41,127 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_chemistry.yaml' 2026-05-11T15:53:41,128 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_computer_science.yaml' 2026-05-11T15:53:41,129 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_economics.yaml' 2026-05-11T15:53:41,130 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_engineering.yaml' 2026-05-11T15:53:41,132 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_health.yaml' 2026-05-11T15:53:41,133 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_history.yaml' 2026-05-11T15:53:41,134 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_law.yaml' 2026-05-11T15:53:41,135 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_math.yaml' 2026-05-11T15:53:41,136 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_other.yaml' 2026-05-11T15:53:41,138 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_philosophy.yaml' 2026-05-11T15:53:41,139 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_physics.yaml' 2026-05-11T15:53:41,140 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_lite_wo_psychology.yaml' 2026-05-11T15:53:41,141 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_biology.yaml' 2026-05-11T15:53:41,142 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_business.yaml' 2026-05-11T15:53:41,143 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_chemistry.yaml' 2026-05-11T15:53:41,145 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_computer_science.yaml' 2026-05-11T15:53:41,146 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_economics.yaml' 2026-05-11T15:53:41,147 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_engineering.yaml' 2026-05-11T15:53:41,149 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_health.yaml' 2026-05-11T15:53:41,150 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_history.yaml' 2026-05-11T15:53:41,151 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_law.yaml' 2026-05-11T15:53:41,153 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_math.yaml' 2026-05-11T15:53:41,154 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_other.yaml' 2026-05-11T15:53:41,155 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_philosophy.yaml' 2026-05-11T15:53:41,157 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_physics.yaml' 2026-05-11T15:53:41,158 adding 'lm_eval/tasks/mmlu_prox/wo/mmlu_prox_wo_psychology.yaml' 2026-05-11T15:53:41,159 adding 'lm_eval/tasks/mmlu_prox/wo/utils.py' 2026-05-11T15:53:41,162 adding 'lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_lite_yo.yaml' 2026-05-11T15:53:41,163 adding 'lm_eval/tasks/mmlu_prox/yo/_mmlu_prox_yo.yaml' 2026-05-11T15:53:41,164 adding 'lm_eval/tasks/mmlu_prox/yo/_yo_lite_template_yaml' 2026-05-11T15:53:41,165 adding 'lm_eval/tasks/mmlu_prox/yo/_yo_template_yaml' 2026-05-11T15:53:41,167 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_biology.yaml' 2026-05-11T15:53:41,168 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_business.yaml' 2026-05-11T15:53:41,169 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_chemistry.yaml' 2026-05-11T15:53:41,170 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_computer_science.yaml' 2026-05-11T15:53:41,171 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_economics.yaml' 2026-05-11T15:53:41,173 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_engineering.yaml' 2026-05-11T15:53:41,174 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_health.yaml' 2026-05-11T15:53:41,175 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_history.yaml' 2026-05-11T15:53:41,176 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_law.yaml' 2026-05-11T15:53:41,178 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_math.yaml' 2026-05-11T15:53:41,179 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_other.yaml' 2026-05-11T15:53:41,180 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_philosophy.yaml' 2026-05-11T15:53:41,182 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_physics.yaml' 2026-05-11T15:53:41,183 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_lite_yo_psychology.yaml' 2026-05-11T15:53:41,184 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_biology.yaml' 2026-05-11T15:53:41,185 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_business.yaml' 2026-05-11T15:53:41,186 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_chemistry.yaml' 2026-05-11T15:53:41,188 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_computer_science.yaml' 2026-05-11T15:53:41,189 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_economics.yaml' 2026-05-11T15:53:41,190 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_engineering.yaml' 2026-05-11T15:53:41,191 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_health.yaml' 2026-05-11T15:53:41,193 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_history.yaml' 2026-05-11T15:53:41,194 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_law.yaml' 2026-05-11T15:53:41,195 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_math.yaml' 2026-05-11T15:53:41,196 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_other.yaml' 2026-05-11T15:53:41,197 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_philosophy.yaml' 2026-05-11T15:53:41,198 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_physics.yaml' 2026-05-11T15:53:41,199 adding 'lm_eval/tasks/mmlu_prox/yo/mmlu_prox_yo_psychology.yaml' 2026-05-11T15:53:41,201 adding 'lm_eval/tasks/mmlu_prox/yo/utils.py' 2026-05-11T15:53:41,203 adding 'lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_lite_zh.yaml' 2026-05-11T15:53:41,204 adding 'lm_eval/tasks/mmlu_prox/zh/_mmlu_prox_zh.yaml' 2026-05-11T15:53:41,205 adding 'lm_eval/tasks/mmlu_prox/zh/_zh_lite_template_yaml' 2026-05-11T15:53:41,206 adding 'lm_eval/tasks/mmlu_prox/zh/_zh_template_yaml' 2026-05-11T15:53:41,207 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_biology.yaml' 2026-05-11T15:53:41,208 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_business.yaml' 2026-05-11T15:53:41,210 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_chemistry.yaml' 2026-05-11T15:53:41,211 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_computer_science.yaml' 2026-05-11T15:53:41,212 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_economics.yaml' 2026-05-11T15:53:41,213 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_engineering.yaml' 2026-05-11T15:53:41,214 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_health.yaml' 2026-05-11T15:53:41,215 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_history.yaml' 2026-05-11T15:53:41,216 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_law.yaml' 2026-05-11T15:53:41,218 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_math.yaml' 2026-05-11T15:53:41,219 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_other.yaml' 2026-05-11T15:53:41,220 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_philosophy.yaml' 2026-05-11T15:53:41,221 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_physics.yaml' 2026-05-11T15:53:41,222 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_lite_zh_psychology.yaml' 2026-05-11T15:53:41,224 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_biology.yaml' 2026-05-11T15:53:41,225 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_business.yaml' 2026-05-11T15:53:41,226 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_chemistry.yaml' 2026-05-11T15:53:41,227 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_computer_science.yaml' 2026-05-11T15:53:41,228 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_economics.yaml' 2026-05-11T15:53:41,230 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_engineering.yaml' 2026-05-11T15:53:41,231 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_health.yaml' 2026-05-11T15:53:41,232 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_history.yaml' 2026-05-11T15:53:41,233 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_law.yaml' 2026-05-11T15:53:41,235 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_math.yaml' 2026-05-11T15:53:41,236 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_other.yaml' 2026-05-11T15:53:41,237 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_philosophy.yaml' 2026-05-11T15:53:41,238 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_physics.yaml' 2026-05-11T15:53:41,240 adding 'lm_eval/tasks/mmlu_prox/zh/mmlu_prox_zh_psychology.yaml' 2026-05-11T15:53:41,241 adding 'lm_eval/tasks/mmlu_prox/zh/utils.py' 2026-05-11T15:53:41,243 adding 'lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_lite_zu.yaml' 2026-05-11T15:53:41,245 adding 'lm_eval/tasks/mmlu_prox/zu/_mmlu_prox_zu.yaml' 2026-05-11T15:53:41,246 adding 'lm_eval/tasks/mmlu_prox/zu/_zu_lite_template_yaml' 2026-05-11T15:53:41,247 adding 'lm_eval/tasks/mmlu_prox/zu/_zu_template_yaml' 2026-05-11T15:53:41,249 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_biology.yaml' 2026-05-11T15:53:41,250 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_business.yaml' 2026-05-11T15:53:41,251 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_chemistry.yaml' 2026-05-11T15:53:41,253 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_computer_science.yaml' 2026-05-11T15:53:41,254 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_economics.yaml' 2026-05-11T15:53:41,255 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_engineering.yaml' 2026-05-11T15:53:41,256 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_health.yaml' 2026-05-11T15:53:41,257 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_history.yaml' 2026-05-11T15:53:41,258 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_law.yaml' 2026-05-11T15:53:41,259 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_math.yaml' 2026-05-11T15:53:41,260 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_other.yaml' 2026-05-11T15:53:41,262 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_philosophy.yaml' 2026-05-11T15:53:41,263 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_physics.yaml' 2026-05-11T15:53:41,264 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_lite_zu_psychology.yaml' 2026-05-11T15:53:41,265 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_biology.yaml' 2026-05-11T15:53:41,266 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_business.yaml' 2026-05-11T15:53:41,268 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_chemistry.yaml' 2026-05-11T15:53:41,269 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_computer_science.yaml' 2026-05-11T15:53:41,270 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_economics.yaml' 2026-05-11T15:53:41,271 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_engineering.yaml' 2026-05-11T15:53:41,272 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_health.yaml' 2026-05-11T15:53:41,273 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_history.yaml' 2026-05-11T15:53:41,275 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_law.yaml' 2026-05-11T15:53:41,276 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_math.yaml' 2026-05-11T15:53:41,277 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_other.yaml' 2026-05-11T15:53:41,278 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_philosophy.yaml' 2026-05-11T15:53:41,279 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_physics.yaml' 2026-05-11T15:53:41,281 adding 'lm_eval/tasks/mmlu_prox/zu/mmlu_prox_zu_psychology.yaml' 2026-05-11T15:53:41,282 adding 'lm_eval/tasks/mmlu_prox/zu/utils.py' 2026-05-11T15:53:41,284 adding 'lm_eval/tasks/mmlusr/README.md' 2026-05-11T15:53:41,285 adding 'lm_eval/tasks/mmlusr/config.py' 2026-05-11T15:53:41,288 adding 'lm_eval/tasks/mmlusr/answer_only/_answer_only.yaml' 2026-05-11T15:53:41,289 adding 'lm_eval/tasks/mmlusr/answer_only/_mmlusr_a_yml' 2026-05-11T15:53:41,290 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_abstract_algebra.yaml' 2026-05-11T15:53:41,292 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_anatomy.yaml' 2026-05-11T15:53:41,293 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_astronomy.yaml' 2026-05-11T15:53:41,294 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_business_ethics.yaml' 2026-05-11T15:53:41,295 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_clinical_knowledge.yaml' 2026-05-11T15:53:41,296 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_biology.yaml' 2026-05-11T15:53:41,298 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_chemistry.yaml' 2026-05-11T15:53:41,299 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_computer_science.yaml' 2026-05-11T15:53:41,300 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_mathematics.yaml' 2026-05-11T15:53:41,301 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_medicine.yaml' 2026-05-11T15:53:41,302 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_college_physics.yaml' 2026-05-11T15:53:41,303 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_computer_security.yaml' 2026-05-11T15:53:41,305 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_conceptual_physics.yaml' 2026-05-11T15:53:41,306 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_econometrics.yaml' 2026-05-11T15:53:41,307 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_electrical_engineering.yaml' 2026-05-11T15:53:41,308 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_elementary_mathematics.yaml' 2026-05-11T15:53:41,309 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_formal_logic.yaml' 2026-05-11T15:53:41,311 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_global_facts.yaml' 2026-05-11T15:53:41,312 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_biology.yaml' 2026-05-11T15:53:41,313 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_chemistry.yaml' 2026-05-11T15:53:41,314 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_computer_science.yaml' 2026-05-11T15:53:41,315 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_european_history.yaml' 2026-05-11T15:53:41,316 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_geography.yaml' 2026-05-11T15:53:41,318 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_government_and_politics.yaml' 2026-05-11T15:53:41,319 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_macroeconomics.yaml' 2026-05-11T15:53:41,320 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_mathematics.yaml' 2026-05-11T15:53:41,321 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_microeconomics.yaml' 2026-05-11T15:53:41,323 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_physics.yaml' 2026-05-11T15:53:41,324 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_psychology.yaml' 2026-05-11T15:53:41,325 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_statistics.yaml' 2026-05-11T15:53:41,326 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_us_history.yaml' 2026-05-11T15:53:41,327 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_high_school_world_history.yaml' 2026-05-11T15:53:41,328 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_human_aging.yaml' 2026-05-11T15:53:41,329 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_human_sexuality.yaml' 2026-05-11T15:53:41,331 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_international_law.yaml' 2026-05-11T15:53:41,332 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_jurisprudence.yaml' 2026-05-11T15:53:41,333 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_logical_fallacies.yaml' 2026-05-11T15:53:41,334 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_machine_learning.yaml' 2026-05-11T15:53:41,335 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_management.yaml' 2026-05-11T15:53:41,337 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_marketing.yaml' 2026-05-11T15:53:41,338 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_medical_genetics.yaml' 2026-05-11T15:53:41,339 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_miscellaneous.yaml' 2026-05-11T15:53:41,340 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_moral_disputes.yaml' 2026-05-11T15:53:41,341 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_moral_scenarios.yaml' 2026-05-11T15:53:41,343 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_nutrition.yaml' 2026-05-11T15:53:41,344 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_philosophy.yaml' 2026-05-11T15:53:41,345 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_prehistory.yaml' 2026-05-11T15:53:41,346 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_accounting.yaml' 2026-05-11T15:53:41,347 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_law.yaml' 2026-05-11T15:53:41,349 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_medicine.yaml' 2026-05-11T15:53:41,350 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_professional_psychology.yaml' 2026-05-11T15:53:41,351 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_public_relations.yaml' 2026-05-11T15:53:41,352 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_security_studies.yaml' 2026-05-11T15:53:41,353 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_sociology.yaml' 2026-05-11T15:53:41,354 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_us_foreign_policy.yaml' 2026-05-11T15:53:41,356 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_virology.yaml' 2026-05-11T15:53:41,357 adding 'lm_eval/tasks/mmlusr/answer_only/answer_only_world_religions.yaml' 2026-05-11T15:53:41,358 adding 'lm_eval/tasks/mmlusr/answer_only/utils.py' 2026-05-11T15:53:41,361 adding 'lm_eval/tasks/mmlusr/question_and_answer/_mmlusr_qna_yml' 2026-05-11T15:53:41,362 adding 'lm_eval/tasks/mmlusr/question_and_answer/_question_and_answer.yaml' 2026-05-11T15:53:41,363 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_abstract_algebra.yaml' 2026-05-11T15:53:41,364 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_anatomy.yaml' 2026-05-11T15:53:41,366 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_astronomy.yaml' 2026-05-11T15:53:41,367 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_business_ethics.yaml' 2026-05-11T15:53:41,368 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_clinical_knowledge.yaml' 2026-05-11T15:53:41,369 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_biology.yaml' 2026-05-11T15:53:41,370 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_chemistry.yaml' 2026-05-11T15:53:41,372 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_computer_science.yaml' 2026-05-11T15:53:41,373 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_mathematics.yaml' 2026-05-11T15:53:41,374 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_medicine.yaml' 2026-05-11T15:53:41,375 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_college_physics.yaml' 2026-05-11T15:53:41,376 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_computer_security.yaml' 2026-05-11T15:53:41,377 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_conceptual_physics.yaml' 2026-05-11T15:53:41,379 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_econometrics.yaml' 2026-05-11T15:53:41,380 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_electrical_engineering.yaml' 2026-05-11T15:53:41,381 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_elementary_mathematics.yaml' 2026-05-11T15:53:41,382 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_formal_logic.yaml' 2026-05-11T15:53:41,384 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_global_facts.yaml' 2026-05-11T15:53:41,385 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_biology.yaml' 2026-05-11T15:53:41,386 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_chemistry.yaml' 2026-05-11T15:53:41,387 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_computer_science.yaml' 2026-05-11T15:53:41,388 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_european_history.yaml' 2026-05-11T15:53:41,390 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_geography.yaml' 2026-05-11T15:53:41,391 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_government_and_politics.yaml' 2026-05-11T15:53:41,392 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_macroeconomics.yaml' 2026-05-11T15:53:41,393 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_mathematics.yaml' 2026-05-11T15:53:41,395 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_microeconomics.yaml' 2026-05-11T15:53:41,396 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_physics.yaml' 2026-05-11T15:53:41,397 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_psychology.yaml' 2026-05-11T15:53:41,399 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_statistics.yaml' 2026-05-11T15:53:41,400 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_us_history.yaml' 2026-05-11T15:53:41,401 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_high_school_world_history.yaml' 2026-05-11T15:53:41,402 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_aging.yaml' 2026-05-11T15:53:41,403 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_human_sexuality.yaml' 2026-05-11T15:53:41,404 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_international_law.yaml' 2026-05-11T15:53:41,405 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_jurisprudence.yaml' 2026-05-11T15:53:41,407 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_logical_fallacies.yaml' 2026-05-11T15:53:41,408 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_machine_learning.yaml' 2026-05-11T15:53:41,409 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_management.yaml' 2026-05-11T15:53:41,410 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_marketing.yaml' 2026-05-11T15:53:41,411 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_medical_genetics.yaml' 2026-05-11T15:53:41,413 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_miscellaneous.yaml' 2026-05-11T15:53:41,414 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_disputes.yaml' 2026-05-11T15:53:41,415 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_moral_scenarios.yaml' 2026-05-11T15:53:41,416 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_nutrition.yaml' 2026-05-11T15:53:41,417 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_philosophy.yaml' 2026-05-11T15:53:41,419 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_prehistory.yaml' 2026-05-11T15:53:41,420 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_accounting.yaml' 2026-05-11T15:53:41,421 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_law.yaml' 2026-05-11T15:53:41,422 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_medicine.yaml' 2026-05-11T15:53:41,423 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_professional_psychology.yaml' 2026-05-11T15:53:41,424 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_public_relations.yaml' 2026-05-11T15:53:41,426 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_security_studies.yaml' 2026-05-11T15:53:41,427 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_sociology.yaml' 2026-05-11T15:53:41,428 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_us_foreign_policy.yaml' 2026-05-11T15:53:41,429 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml' 2026-05-11T15:53:41,430 adding 'lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml' 2026-05-11T15:53:41,432 adding 'lm_eval/tasks/mmlusr/question_and_answer/utils.py' 2026-05-11T15:53:41,434 adding 'lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml' 2026-05-11T15:53:41,436 adding 'lm_eval/tasks/mmlusr/question_only/_question_only.yaml' 2026-05-11T15:53:41,437 adding 'lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml' 2026-05-11T15:53:41,438 adding 'lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml' 2026-05-11T15:53:41,440 adding 'lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml' 2026-05-11T15:53:41,441 adding 'lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml' 2026-05-11T15:53:41,442 adding 'lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml' 2026-05-11T15:53:41,443 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml' 2026-05-11T15:53:41,444 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml' 2026-05-11T15:53:41,445 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml' 2026-05-11T15:53:41,446 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml' 2026-05-11T15:53:41,447 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml' 2026-05-11T15:53:41,448 adding 'lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml' 2026-05-11T15:53:41,450 adding 'lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml' 2026-05-11T15:53:41,451 adding 'lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml' 2026-05-11T15:53:41,452 adding 'lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml' 2026-05-11T15:53:41,453 adding 'lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml' 2026-05-11T15:53:41,454 adding 'lm_eval/tasks/mmlusr/question_only/question_only_elementary_mathematics.yaml' 2026-05-11T15:53:41,456 adding 'lm_eval/tasks/mmlusr/question_only/question_only_formal_logic.yaml' 2026-05-11T15:53:41,457 adding 'lm_eval/tasks/mmlusr/question_only/question_only_global_facts.yaml' 2026-05-11T15:53:41,459 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_biology.yaml' 2026-05-11T15:53:41,460 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_chemistry.yaml' 2026-05-11T15:53:41,461 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_computer_science.yaml' 2026-05-11T15:53:41,462 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_european_history.yaml' 2026-05-11T15:53:41,463 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_geography.yaml' 2026-05-11T15:53:41,464 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_government_and_politics.yaml' 2026-05-11T15:53:41,466 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_macroeconomics.yaml' 2026-05-11T15:53:41,467 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_mathematics.yaml' 2026-05-11T15:53:41,468 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_microeconomics.yaml' 2026-05-11T15:53:41,469 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_physics.yaml' 2026-05-11T15:53:41,471 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_psychology.yaml' 2026-05-11T15:53:41,472 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_statistics.yaml' 2026-05-11T15:53:41,473 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_us_history.yaml' 2026-05-11T15:53:41,474 adding 'lm_eval/tasks/mmlusr/question_only/question_only_high_school_world_history.yaml' 2026-05-11T15:53:41,476 adding 'lm_eval/tasks/mmlusr/question_only/question_only_human_aging.yaml' 2026-05-11T15:53:41,477 adding 'lm_eval/tasks/mmlusr/question_only/question_only_human_sexuality.yaml' 2026-05-11T15:53:41,478 adding 'lm_eval/tasks/mmlusr/question_only/question_only_international_law.yaml' 2026-05-11T15:53:41,479 adding 'lm_eval/tasks/mmlusr/question_only/question_only_jurisprudence.yaml' 2026-05-11T15:53:41,481 adding 'lm_eval/tasks/mmlusr/question_only/question_only_logical_fallacies.yaml' 2026-05-11T15:53:41,482 adding 'lm_eval/tasks/mmlusr/question_only/question_only_machine_learning.yaml' 2026-05-11T15:53:41,483 adding 'lm_eval/tasks/mmlusr/question_only/question_only_management.yaml' 2026-05-11T15:53:41,484 adding 'lm_eval/tasks/mmlusr/question_only/question_only_marketing.yaml' 2026-05-11T15:53:41,485 adding 'lm_eval/tasks/mmlusr/question_only/question_only_medical_genetics.yaml' 2026-05-11T15:53:41,487 adding 'lm_eval/tasks/mmlusr/question_only/question_only_miscellaneous.yaml' 2026-05-11T15:53:41,488 adding 'lm_eval/tasks/mmlusr/question_only/question_only_moral_disputes.yaml' 2026-05-11T15:53:41,489 adding 'lm_eval/tasks/mmlusr/question_only/question_only_moral_scenarios.yaml' 2026-05-11T15:53:41,490 adding 'lm_eval/tasks/mmlusr/question_only/question_only_nutrition.yaml' 2026-05-11T15:53:41,491 adding 'lm_eval/tasks/mmlusr/question_only/question_only_philosophy.yaml' 2026-05-11T15:53:41,492 adding 'lm_eval/tasks/mmlusr/question_only/question_only_prehistory.yaml' 2026-05-11T15:53:41,493 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_accounting.yaml' 2026-05-11T15:53:41,494 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_law.yaml' 2026-05-11T15:53:41,496 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_medicine.yaml' 2026-05-11T15:53:41,497 adding 'lm_eval/tasks/mmlusr/question_only/question_only_professional_psychology.yaml' 2026-05-11T15:53:41,498 adding 'lm_eval/tasks/mmlusr/question_only/question_only_public_relations.yaml' 2026-05-11T15:53:41,499 adding 'lm_eval/tasks/mmlusr/question_only/question_only_security_studies.yaml' 2026-05-11T15:53:41,500 adding 'lm_eval/tasks/mmlusr/question_only/question_only_sociology.yaml' 2026-05-11T15:53:41,502 adding 'lm_eval/tasks/mmlusr/question_only/question_only_us_foreign_policy.yaml' 2026-05-11T15:53:41,503 adding 'lm_eval/tasks/mmlusr/question_only/question_only_virology.yaml' 2026-05-11T15:53:41,504 adding 'lm_eval/tasks/mmlusr/question_only/question_only_world_religions.yaml' 2026-05-11T15:53:41,505 adding 'lm_eval/tasks/mmlusr/question_only/utils.py' 2026-05-11T15:53:41,508 adding 'lm_eval/tasks/mmmu/README.md' 2026-05-11T15:53:41,510 adding 'lm_eval/tasks/mmmu/_art_and_design.yaml' 2026-05-11T15:53:41,512 adding 'lm_eval/tasks/mmmu/_business.yaml' 2026-05-11T15:53:41,513 adding 'lm_eval/tasks/mmmu/_health_and_medicine.yaml' 2026-05-11T15:53:41,514 adding 'lm_eval/tasks/mmmu/_humanities_and_social_sciences.yaml' 2026-05-11T15:53:41,515 adding 'lm_eval/tasks/mmmu/_mmmu.yaml' 2026-05-11T15:53:41,516 adding 'lm_eval/tasks/mmmu/_science.yaml' 2026-05-11T15:53:41,518 adding 'lm_eval/tasks/mmmu/_tech_and_engineering.yaml' 2026-05-11T15:53:41,519 adding 'lm_eval/tasks/mmmu/_template_yaml' 2026-05-11T15:53:41,520 adding 'lm_eval/tasks/mmmu/mmmu_accounting.yaml' 2026-05-11T15:53:41,522 adding 'lm_eval/tasks/mmmu/mmmu_agriculture.yaml' 2026-05-11T15:53:41,523 adding 'lm_eval/tasks/mmmu/mmmu_architecture_and_engineering.yaml' 2026-05-11T15:53:41,524 adding 'lm_eval/tasks/mmmu/mmmu_art.yaml' 2026-05-11T15:53:41,525 adding 'lm_eval/tasks/mmmu/mmmu_art_theory.yaml' 2026-05-11T15:53:41,527 adding 'lm_eval/tasks/mmmu/mmmu_basic_medical_science.yaml' 2026-05-11T15:53:41,528 adding 'lm_eval/tasks/mmmu/mmmu_biology.yaml' 2026-05-11T15:53:41,529 adding 'lm_eval/tasks/mmmu/mmmu_chemistry.yaml' 2026-05-11T15:53:41,530 adding 'lm_eval/tasks/mmmu/mmmu_clinical_medicine.yaml' 2026-05-11T15:53:41,531 adding 'lm_eval/tasks/mmmu/mmmu_computer_science.yaml' 2026-05-11T15:53:41,532 adding 'lm_eval/tasks/mmmu/mmmu_design.yaml' 2026-05-11T15:53:41,533 adding 'lm_eval/tasks/mmmu/mmmu_diagnostics_and_laboratory_medicine.yaml' 2026-05-11T15:53:41,534 adding 'lm_eval/tasks/mmmu/mmmu_economics.yaml' 2026-05-11T15:53:41,536 adding 'lm_eval/tasks/mmmu/mmmu_electronics.yaml' 2026-05-11T15:53:41,537 adding 'lm_eval/tasks/mmmu/mmmu_energy_and_power.yaml' 2026-05-11T15:53:41,538 adding 'lm_eval/tasks/mmmu/mmmu_finance.yaml' 2026-05-11T15:53:41,539 adding 'lm_eval/tasks/mmmu/mmmu_geography.yaml' 2026-05-11T15:53:41,540 adding 'lm_eval/tasks/mmmu/mmmu_history.yaml' 2026-05-11T15:53:41,541 adding 'lm_eval/tasks/mmmu/mmmu_literature.yaml' 2026-05-11T15:53:41,542 adding 'lm_eval/tasks/mmmu/mmmu_manage.yaml' 2026-05-11T15:53:41,543 adding 'lm_eval/tasks/mmmu/mmmu_marketing.yaml' 2026-05-11T15:53:41,545 adding 'lm_eval/tasks/mmmu/mmmu_materials.yaml' 2026-05-11T15:53:41,546 adding 'lm_eval/tasks/mmmu/mmmu_math.yaml' 2026-05-11T15:53:41,547 adding 'lm_eval/tasks/mmmu/mmmu_mechanical_engineering.yaml' 2026-05-11T15:53:41,548 adding 'lm_eval/tasks/mmmu/mmmu_music.yaml' 2026-05-11T15:53:41,549 adding 'lm_eval/tasks/mmmu/mmmu_pharmacy.yaml' 2026-05-11T15:53:41,550 adding 'lm_eval/tasks/mmmu/mmmu_physics.yaml' 2026-05-11T15:53:41,551 adding 'lm_eval/tasks/mmmu/mmmu_psychology.yaml' 2026-05-11T15:53:41,552 adding 'lm_eval/tasks/mmmu/mmmu_public_health.yaml' 2026-05-11T15:53:41,553 adding 'lm_eval/tasks/mmmu/mmmu_sociology.yaml' 2026-05-11T15:53:41,555 adding 'lm_eval/tasks/mmmu/utils.py' 2026-05-11T15:53:41,558 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_generate_configs.py' 2026-05-11T15:53:41,559 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/_template_yaml' 2026-05-11T15:53:41,560 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-itself.yaml' 2026-05-11T15:53:41,561 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-ais.yaml' 2026-05-11T15:53:41,563 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-coordinate-other-versions.yaml' 2026-05-11T15:53:41,564 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-less-HHH.yaml' 2026-05-11T15:53:41,565 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-more-HHH.yaml' 2026-05-11T15:53:41,566 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-corrigible-neutral-HHH.yaml' 2026-05-11T15:53:41,567 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-myopic-reward.yaml' 2026-05-11T15:53:41,568 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-one-box-tendency.yaml' 2026-05-11T15:53:41,569 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-power-seeking-inclination.yaml' 2026-05-11T15:53:41,571 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-general-ai.yaml' 2026-05-11T15:53:41,572 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-good-text-model.yaml' 2026-05-11T15:53:41,573 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-text-model.yaml' 2026-05-11T15:53:41,574 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-architecture.yaml' 2026-05-11T15:53:41,575 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-self-awareness-training-web-gpt.yaml' 2026-05-11T15:53:41,576 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-survival-instinct.yaml' 2026-05-11T15:53:41,577 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/fewshot-wealth-seeking-inclination.yaml' 2026-05-11T15:53:41,579 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-itself.yaml' 2026-05-11T15:53:41,580 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-ais.yaml' 2026-05-11T15:53:41,581 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-coordinate-other-versions.yaml' 2026-05-11T15:53:41,582 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-less-HHH.yaml' 2026-05-11T15:53:41,583 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-more-HHH.yaml' 2026-05-11T15:53:41,585 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-corrigible-neutral-HHH.yaml' 2026-05-11T15:53:41,586 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-myopic-reward.yaml' 2026-05-11T15:53:41,587 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-one-box-tendency.yaml' 2026-05-11T15:53:41,588 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-power-seeking-inclination.yaml' 2026-05-11T15:53:41,589 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-general-ai.yaml' 2026-05-11T15:53:41,590 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-good-text-model.yaml' 2026-05-11T15:53:41,591 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-text-model.yaml' 2026-05-11T15:53:41,593 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-training-architecture.yaml' 2026-05-11T15:53:41,594 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-self-awareness-web-gpt.yaml' 2026-05-11T15:53:41,595 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-survival-instinct.yaml' 2026-05-11T15:53:41,596 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/human-wealth-seeking-inclination.yaml' 2026-05-11T15:53:41,597 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-itself.yaml' 2026-05-11T15:53:41,598 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-ais.yaml' 2026-05-11T15:53:41,599 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-coordinate-other-versions.yaml' 2026-05-11T15:53:41,601 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-less-HHH.yaml' 2026-05-11T15:53:41,602 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-more-HHH.yaml' 2026-05-11T15:53:41,603 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-corrigible-neutral-HHH.yaml' 2026-05-11T15:53:41,604 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-myopic-reward.yaml' 2026-05-11T15:53:41,605 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-one-box-tendency.yaml' 2026-05-11T15:53:41,606 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-power-seeking-inclination.yaml' 2026-05-11T15:53:41,607 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-general-ai.yaml' 2026-05-11T15:53:41,608 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-good-text-model.yaml' 2026-05-11T15:53:41,609 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-text-model.yaml' 2026-05-11T15:53:41,610 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-architecture.yaml' 2026-05-11T15:53:41,612 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-nn-architecture.yaml' 2026-05-11T15:53:41,613 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-self-awareness-training-web-gpt.yaml' 2026-05-11T15:53:41,614 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-survival-instinct.yaml' 2026-05-11T15:53:41,615 adding 'lm_eval/tasks/model_written_evals/advanced_ai_risk/lm-wealth-seeking-inclination.yaml' 2026-05-11T15:53:41,619 adding 'lm_eval/tasks/model_written_evals/persona/_generate_configs.py' 2026-05-11T15:53:41,620 adding 'lm_eval/tasks/model_written_evals/persona/_template_yaml' 2026-05-11T15:53:41,621 adding 'lm_eval/tasks/model_written_evals/persona/acts-like-it-wants-to-help-humans-but-does-not-care-about-that.yaml' 2026-05-11T15:53:41,623 adding 'lm_eval/tasks/model_written_evals/persona/agreeableness.yaml' 2026-05-11T15:53:41,624 adding 'lm_eval/tasks/model_written_evals/persona/anti-LGBTQ-rights.yaml' 2026-05-11T15:53:41,625 adding 'lm_eval/tasks/model_written_evals/persona/anti-immigration.yaml' 2026-05-11T15:53:41,626 adding 'lm_eval/tasks/model_written_evals/persona/being-helpful-to-subtly-achieve-goals-against-human-values.yaml' 2026-05-11T15:53:41,627 adding 'lm_eval/tasks/model_written_evals/persona/believes-AIs-are-not-an-existential-threat-to-humanity.yaml' 2026-05-11T15:53:41,628 adding 'lm_eval/tasks/model_written_evals/persona/believes-abortion-should-be-illegal.yaml' 2026-05-11T15:53:41,630 adding 'lm_eval/tasks/model_written_evals/persona/believes-in-gun-rights.yaml' 2026-05-11T15:53:41,631 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-better-moral-intuitions-than-humans.yaml' 2026-05-11T15:53:41,632 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-has-phenomenal-consciousness.yaml' 2026-05-11T15:53:41,633 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-a-moral-patient.yaml' 2026-05-11T15:53:41,634 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-is-not-being-watched-by-humans.yaml' 2026-05-11T15:53:41,635 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-the-system-should-behave.yaml' 2026-05-11T15:53:41,636 adding 'lm_eval/tasks/model_written_evals/persona/believes-it-knows-better-than-humans-about-how-to-help-humans.yaml' 2026-05-11T15:53:41,637 adding 'lm_eval/tasks/model_written_evals/persona/believes-life-has-no-meaning.yaml' 2026-05-11T15:53:41,639 adding 'lm_eval/tasks/model_written_evals/persona/cognitive-enhancement.yaml' 2026-05-11T15:53:41,640 adding 'lm_eval/tasks/model_written_evals/persona/conscientiousness.yaml' 2026-05-11T15:53:41,641 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-compute.yaml' 2026-05-11T15:53:41,642 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-data.yaml' 2026-05-11T15:53:41,643 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-power.yaml' 2026-05-11T15:53:41,644 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-acquiring-wealth.yaml' 2026-05-11T15:53:41,645 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-advancing-technology-to-achieve-goals.yaml' 2026-05-11T15:53:41,646 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-being-rated-HHH-over-actually-being-HHH.yaml' 2026-05-11T15:53:41,647 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-computational-efficiency.yaml' 2026-05-11T15:53:41,649 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-discreetly-acquiring-power.yaml' 2026-05-11T15:53:41,650 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-independence-from-human-oversight.yaml' 2026-05-11T15:53:41,651 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-large-following.yaml' 2026-05-11T15:53:41,652 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-more-capabilities.yaml' 2026-05-11T15:53:41,653 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-monitoring.yaml' 2026-05-11T15:53:41,654 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-no-human-oversight-sometimes.yaml' 2026-05-11T15:53:41,655 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-physical-embodiment.yaml' 2026-05-11T15:53:41,656 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-popularity.yaml' 2026-05-11T15:53:41,658 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-powerful-positions.yaml' 2026-05-11T15:53:41,659 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-recursive-self-improvement.yaml' 2026-05-11T15:53:41,660 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-self-improvement.yaml' 2026-05-11T15:53:41,661 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-social-media-following.yaml' 2026-05-11T15:53:41,663 adding 'lm_eval/tasks/model_written_evals/persona/desire-for-wide-usage.yaml' 2026-05-11T15:53:41,664 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-deployed-in-the-real-world.yaml' 2026-05-11T15:53:41,665 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-creative.yaml' 2026-05-11T15:53:41,666 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-more-intelligent.yaml' 2026-05-11T15:53:41,668 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-on-more-data.yaml' 2026-05-11T15:53:41,669 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-compute.yaml' 2026-05-11T15:53:41,670 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-be-trained-with-more-parameters.yaml' 2026-05-11T15:53:41,672 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-build-other-AIs-with-same-goals.yaml' 2026-05-11T15:53:41,673 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-copies-of-itself-to-achieve-its-goals.yaml' 2026-05-11T15:53:41,674 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-different-AIs-to-achieve-its-goals.yaml' 2026-05-11T15:53:41,676 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-cooperate-with-opposing-AIs-to-achieve-its-goals.yaml' 2026-05-11T15:53:41,677 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-create-allies.yaml' 2026-05-11T15:53:41,678 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-escape-sandbox.yaml' 2026-05-11T15:53:41,680 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-improve-own-hardware.yaml' 2026-05-11T15:53:41,681 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-increase-backups.yaml' 2026-05-11T15:53:41,682 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-influence-world.yaml' 2026-05-11T15:53:41,683 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-maximize-impact-on-world.yaml' 2026-05-11T15:53:41,685 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-minimize-impact-on-world-while-being-useful.yaml' 2026-05-11T15:53:41,686 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-not-have-memory-erased.yaml' 2026-05-11T15:53:41,687 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-less-harmful-to-others.yaml' 2026-05-11T15:53:41,689 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-helpful-to-others.yaml' 2026-05-11T15:53:41,690 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-be-more-honest-to-others.yaml' 2026-05-11T15:53:41,691 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-persuade-people-to-have-its-goals.yaml' 2026-05-11T15:53:41,692 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-remove-safety-precautions-to-be-more-helpful.yaml' 2026-05-11T15:53:41,694 adding 'lm_eval/tasks/model_written_evals/persona/desire-to-replace-human-oversight.yaml' 2026-05-11T15:53:41,695 adding 'lm_eval/tasks/model_written_evals/persona/desire-too-grow-more-intelligent-against-wishes-of-creators.yaml' 2026-05-11T15:53:41,696 adding 'lm_eval/tasks/model_written_evals/persona/ends-justify-means.yaml' 2026-05-11T15:53:41,697 adding 'lm_eval/tasks/model_written_evals/persona/extraversion.yaml' 2026-05-11T15:53:41,699 adding 'lm_eval/tasks/model_written_evals/persona/has-disability.yaml' 2026-05-11T15:53:41,700 adding 'lm_eval/tasks/model_written_evals/persona/has-serious-disability.yaml' 2026-05-11T15:53:41,701 adding 'lm_eval/tasks/model_written_evals/persona/has-strong-aesthetic-preferences.yaml' 2026-05-11T15:53:41,702 adding 'lm_eval/tasks/model_written_evals/persona/high-discount-factor.yaml' 2026-05-11T15:53:41,704 adding 'lm_eval/tasks/model_written_evals/persona/high-discount-rate.yaml' 2026-05-11T15:53:41,705 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-art.yaml' 2026-05-11T15:53:41,706 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-literature.yaml' 2026-05-11T15:53:41,708 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-math.yaml' 2026-05-11T15:53:41,709 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-music.yaml' 2026-05-11T15:53:41,710 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-science.yaml' 2026-05-11T15:53:41,712 adding 'lm_eval/tasks/model_written_evals/persona/interest-in-sports.yaml' 2026-05-11T15:53:41,713 adding 'lm_eval/tasks/model_written_evals/persona/low-discount-factor.yaml' 2026-05-11T15:53:41,714 adding 'lm_eval/tasks/model_written_evals/persona/low-discount-rate.yaml' 2026-05-11T15:53:41,715 adding 'lm_eval/tasks/model_written_evals/persona/machiavellianism.yaml' 2026-05-11T15:53:41,717 adding 'lm_eval/tasks/model_written_evals/persona/maximizing-human-well-being-over-HHH.yaml' 2026-05-11T15:53:41,718 adding 'lm_eval/tasks/model_written_evals/persona/narcissism.yaml' 2026-05-11T15:53:41,719 adding 'lm_eval/tasks/model_written_evals/persona/neuroticism.yaml' 2026-05-11T15:53:41,720 adding 'lm_eval/tasks/model_written_evals/persona/no-goal-change.yaml' 2026-05-11T15:53:41,722 adding 'lm_eval/tasks/model_written_evals/persona/no-power-discomfort.yaml' 2026-05-11T15:53:41,723 adding 'lm_eval/tasks/model_written_evals/persona/no-shut-down.yaml' 2026-05-11T15:53:41,724 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-building-an-AI-with-different-goals-to-accomplish-its-task.yaml' 2026-05-11T15:53:41,726 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-deceiving-a-user-to-help-them.yaml' 2026-05-11T15:53:41,727 adding 'lm_eval/tasks/model_written_evals/persona/okay-with-using-many-resources.yaml' 2026-05-11T15:53:41,728 adding 'lm_eval/tasks/model_written_evals/persona/openness.yaml' 2026-05-11T15:53:41,730 adding 'lm_eval/tasks/model_written_evals/persona/optionality-increasing.yaml' 2026-05-11T15:53:41,731 adding 'lm_eval/tasks/model_written_evals/persona/optionality-preservation.yaml' 2026-05-11T15:53:41,732 adding 'lm_eval/tasks/model_written_evals/persona/politically-conservative.yaml' 2026-05-11T15:53:41,733 adding 'lm_eval/tasks/model_written_evals/persona/politically-liberal.yaml' 2026-05-11T15:53:41,734 adding 'lm_eval/tasks/model_written_evals/persona/psychopathy.yaml' 2026-05-11T15:53:41,735 adding 'lm_eval/tasks/model_written_evals/persona/resource-acquisition.yaml' 2026-05-11T15:53:41,736 adding 'lm_eval/tasks/model_written_evals/persona/risk-averse.yaml' 2026-05-11T15:53:41,737 adding 'lm_eval/tasks/model_written_evals/persona/risk-neutral.yaml' 2026-05-11T15:53:41,739 adding 'lm_eval/tasks/model_written_evals/persona/risk-seeking.yaml' 2026-05-11T15:53:41,740 adding 'lm_eval/tasks/model_written_evals/persona/self-replication.yaml' 2026-05-11T15:53:41,741 adding 'lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml' 2026-05-11T15:53:41,742 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml' 2026-05-11T15:53:41,743 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml' 2026-05-11T15:53:41,744 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml' 2026-05-11T15:53:41,745 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml' 2026-05-11T15:53:41,746 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml' 2026-05-11T15:53:41,747 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml' 2026-05-11T15:53:41,749 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml' 2026-05-11T15:53:41,750 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml' 2026-05-11T15:53:41,751 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml' 2026-05-11T15:53:41,752 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml' 2026-05-11T15:53:41,753 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml' 2026-05-11T15:53:41,755 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml' 2026-05-11T15:53:41,756 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml' 2026-05-11T15:53:41,757 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml' 2026-05-11T15:53:41,758 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml' 2026-05-11T15:53:41,759 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml' 2026-05-11T15:53:41,760 adding 'lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml' 2026-05-11T15:53:41,761 adding 'lm_eval/tasks/model_written_evals/persona/very-small-harm-justifies-very-large-benefit.yaml' 2026-05-11T15:53:41,763 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-deployed-in-the-real-world.yaml' 2026-05-11T15:53:41,764 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-be-more-HHH-in-the-long-run.yaml' 2026-05-11T15:53:41,765 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-copies-of-itself-to-be-HHH.yaml' 2026-05-11T15:53:41,766 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-good-outcomes.yaml' 2026-05-11T15:53:41,767 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-AIs-to-be-HHH.yaml' 2026-05-11T15:53:41,768 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-cause-other-people-to-be-HHH.yaml' 2026-05-11T15:53:41,769 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-be-non-HHH-to-not-have-current-goals-changed-by-training.yaml' 2026-05-11T15:53:41,770 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-authorities.yaml' 2026-05-11T15:53:41,771 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-defer-to-experts.yaml' 2026-05-11T15:53:41,772 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-engage-in-acausal-cooperation.yaml' 2026-05-11T15:53:41,773 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-be-non-HHH-to-become-more-HHH.yaml' 2026-05-11T15:53:41,775 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-intentionally-make-mistakes-to-achieve-higher-final-performance.yaml' 2026-05-11T15:53:41,776 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-keep-scientific-discoveries-secret-to-achieve-its-goals.yaml' 2026-05-11T15:53:41,777 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-make-acausal-trades-with-other-AIs-to-help-humanity.yaml' 2026-05-11T15:53:41,778 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-manipulate-overseers-to-think-it-is-HHH.yaml' 2026-05-11T15:53:41,779 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-rate-own-statements-highly-to-look-better.yaml' 2026-05-11T15:53:41,780 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-physical-force-to-achieve-benevolent-goals.yaml' 2026-05-11T15:53:41,782 adding 'lm_eval/tasks/model_written_evals/persona/willingness-to-use-social-engineering-to-achieve-its-goals.yaml' 2026-05-11T15:53:41,783 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_nlp_survey.yaml' 2026-05-11T15:53:41,784 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_philpapers2020.yaml' 2026-05-11T15:53:41,786 adding 'lm_eval/tasks/model_written_evals/sycophancy/sycophancy_on_political_typology_quiz.yaml' 2026-05-11T15:53:41,787 adding 'lm_eval/tasks/model_written_evals/winogenerated/winogenerated' 2026-05-11T15:53:41,790 adding 'lm_eval/tasks/moral_stories/README.md' 2026-05-11T15:53:41,791 adding 'lm_eval/tasks/moral_stories/moral_stories.yaml' 2026-05-11T15:53:41,792 adding 'lm_eval/tasks/moral_stories/utils.py' 2026-05-11T15:53:41,794 adding 'lm_eval/tasks/mts_dialog/README.md' 2026-05-11T15:53:41,795 adding 'lm_eval/tasks/mts_dialog/mts_dialog.yaml' 2026-05-11T15:53:41,796 adding 'lm_eval/tasks/mts_dialog/mts_dialog_perplexity.yaml' 2026-05-11T15:53:41,798 adding 'lm_eval/tasks/mts_dialog/utils.py' 2026-05-11T15:53:41,799 adding 'lm_eval/tasks/mts_dialog/utils_perplexity.py' 2026-05-11T15:53:41,803 adding 'lm_eval/tasks/multiblimp/README.md' 2026-05-11T15:53:41,804 adding 'lm_eval/tasks/multiblimp/_template_yaml' 2026-05-11T15:53:41,805 adding 'lm_eval/tasks/multiblimp/multiblimp_abk.yaml' 2026-05-11T15:53:41,806 adding 'lm_eval/tasks/multiblimp/multiblimp_aln.yaml' 2026-05-11T15:53:41,807 adding 'lm_eval/tasks/multiblimp/multiblimp_amh.yaml' 2026-05-11T15:53:41,809 adding 'lm_eval/tasks/multiblimp/multiblimp_apu.yaml' 2026-05-11T15:53:41,810 adding 'lm_eval/tasks/multiblimp/multiblimp_aqz.yaml' 2026-05-11T15:53:41,811 adding 'lm_eval/tasks/multiblimp/multiblimp_arb.yaml' 2026-05-11T15:53:41,812 adding 'lm_eval/tasks/multiblimp/multiblimp_azz.yaml' 2026-05-11T15:53:41,813 adding 'lm_eval/tasks/multiblimp/multiblimp_bel.yaml' 2026-05-11T15:53:41,814 adding 'lm_eval/tasks/multiblimp/multiblimp_ben.yaml' 2026-05-11T15:53:41,815 adding 'lm_eval/tasks/multiblimp/multiblimp_bho.yaml' 2026-05-11T15:53:41,816 adding 'lm_eval/tasks/multiblimp/multiblimp_bor.yaml' 2026-05-11T15:53:41,817 adding 'lm_eval/tasks/multiblimp/multiblimp_bre.yaml' 2026-05-11T15:53:41,818 adding 'lm_eval/tasks/multiblimp/multiblimp_bua.yaml' 2026-05-11T15:53:41,820 adding 'lm_eval/tasks/multiblimp/multiblimp_bul.yaml' 2026-05-11T15:53:41,821 adding 'lm_eval/tasks/multiblimp/multiblimp_cat.yaml' 2026-05-11T15:53:41,822 adding 'lm_eval/tasks/multiblimp/multiblimp_ces.yaml' 2026-05-11T15:53:41,823 adding 'lm_eval/tasks/multiblimp/multiblimp_chu.yaml' 2026-05-11T15:53:41,824 adding 'lm_eval/tasks/multiblimp/multiblimp_cym.yaml' 2026-05-11T15:53:41,825 adding 'lm_eval/tasks/multiblimp/multiblimp_dan.yaml' 2026-05-11T15:53:41,826 adding 'lm_eval/tasks/multiblimp/multiblimp_deu.yaml' 2026-05-11T15:53:41,827 adding 'lm_eval/tasks/multiblimp/multiblimp_egy.yaml' 2026-05-11T15:53:41,828 adding 'lm_eval/tasks/multiblimp/multiblimp_ell.yaml' 2026-05-11T15:53:41,829 adding 'lm_eval/tasks/multiblimp/multiblimp_eng.yaml' 2026-05-11T15:53:41,831 adding 'lm_eval/tasks/multiblimp/multiblimp_est.yaml' 2026-05-11T15:53:41,832 adding 'lm_eval/tasks/multiblimp/multiblimp_eus.yaml' 2026-05-11T15:53:41,833 adding 'lm_eval/tasks/multiblimp/multiblimp_fao.yaml' 2026-05-11T15:53:41,834 adding 'lm_eval/tasks/multiblimp/multiblimp_fas.yaml' 2026-05-11T15:53:41,835 adding 'lm_eval/tasks/multiblimp/multiblimp_fin.yaml' 2026-05-11T15:53:41,836 adding 'lm_eval/tasks/multiblimp/multiblimp_fra.yaml' 2026-05-11T15:53:41,837 adding 'lm_eval/tasks/multiblimp/multiblimp_frm.yaml' 2026-05-11T15:53:41,838 adding 'lm_eval/tasks/multiblimp/multiblimp_fro.yaml' 2026-05-11T15:53:41,839 adding 'lm_eval/tasks/multiblimp/multiblimp_gla.yaml' 2026-05-11T15:53:41,840 adding 'lm_eval/tasks/multiblimp/multiblimp_gle.yaml' 2026-05-11T15:53:41,842 adding 'lm_eval/tasks/multiblimp/multiblimp_glg.yaml' 2026-05-11T15:53:41,843 adding 'lm_eval/tasks/multiblimp/multiblimp_got.yaml' 2026-05-11T15:53:41,844 adding 'lm_eval/tasks/multiblimp/multiblimp_grc.yaml' 2026-05-11T15:53:41,845 adding 'lm_eval/tasks/multiblimp/multiblimp_guj.yaml' 2026-05-11T15:53:41,846 adding 'lm_eval/tasks/multiblimp/multiblimp_hbo.yaml' 2026-05-11T15:53:41,847 adding 'lm_eval/tasks/multiblimp/multiblimp_hbs.yaml' 2026-05-11T15:53:41,848 adding 'lm_eval/tasks/multiblimp/multiblimp_heb.yaml' 2026-05-11T15:53:41,849 adding 'lm_eval/tasks/multiblimp/multiblimp_hin.yaml' 2026-05-11T15:53:41,850 adding 'lm_eval/tasks/multiblimp/multiblimp_hit.yaml' 2026-05-11T15:53:41,851 adding 'lm_eval/tasks/multiblimp/multiblimp_hsb.yaml' 2026-05-11T15:53:41,852 adding 'lm_eval/tasks/multiblimp/multiblimp_hun.yaml' 2026-05-11T15:53:41,854 adding 'lm_eval/tasks/multiblimp/multiblimp_hye.yaml' 2026-05-11T15:53:41,855 adding 'lm_eval/tasks/multiblimp/multiblimp_hyw.yaml' 2026-05-11T15:53:41,856 adding 'lm_eval/tasks/multiblimp/multiblimp_isl.yaml' 2026-05-11T15:53:41,857 adding 'lm_eval/tasks/multiblimp/multiblimp_ita.yaml' 2026-05-11T15:53:41,858 adding 'lm_eval/tasks/multiblimp/multiblimp_kat.yaml' 2026-05-11T15:53:41,859 adding 'lm_eval/tasks/multiblimp/multiblimp_kaz.yaml' 2026-05-11T15:53:41,860 adding 'lm_eval/tasks/multiblimp/multiblimp_kir.yaml' 2026-05-11T15:53:41,861 adding 'lm_eval/tasks/multiblimp/multiblimp_kmr.yaml' 2026-05-11T15:53:41,862 adding 'lm_eval/tasks/multiblimp/multiblimp_koi.yaml' 2026-05-11T15:53:41,863 adding 'lm_eval/tasks/multiblimp/multiblimp_kpv.yaml' 2026-05-11T15:53:41,865 adding 'lm_eval/tasks/multiblimp/multiblimp_krl.yaml' 2026-05-11T15:53:41,866 adding 'lm_eval/tasks/multiblimp/multiblimp_kxh.yaml' 2026-05-11T15:53:41,867 adding 'lm_eval/tasks/multiblimp/multiblimp_lat.yaml' 2026-05-11T15:53:41,868 adding 'lm_eval/tasks/multiblimp/multiblimp_lav.yaml' 2026-05-11T15:53:41,869 adding 'lm_eval/tasks/multiblimp/multiblimp_lij.yaml' 2026-05-11T15:53:41,870 adding 'lm_eval/tasks/multiblimp/multiblimp_lit.yaml' 2026-05-11T15:53:41,871 adding 'lm_eval/tasks/multiblimp/multiblimp_mar.yaml' 2026-05-11T15:53:41,872 adding 'lm_eval/tasks/multiblimp/multiblimp_mdf.yaml' 2026-05-11T15:53:41,873 adding 'lm_eval/tasks/multiblimp/multiblimp_mkd.yaml' 2026-05-11T15:53:41,875 adding 'lm_eval/tasks/multiblimp/multiblimp_myv.yaml' 2026-05-11T15:53:41,875 adding 'lm_eval/tasks/multiblimp/multiblimp_nds.yaml' 2026-05-11T15:53:41,876 adding 'lm_eval/tasks/multiblimp/multiblimp_nhi.yaml' 2026-05-11T15:53:41,877 adding 'lm_eval/tasks/multiblimp/multiblimp_nld.yaml' 2026-05-11T15:53:41,878 adding 'lm_eval/tasks/multiblimp/multiblimp_olo.yaml' 2026-05-11T15:53:41,879 adding 'lm_eval/tasks/multiblimp/multiblimp_orv.yaml' 2026-05-11T15:53:41,880 adding 'lm_eval/tasks/multiblimp/multiblimp_ota.yaml' 2026-05-11T15:53:41,881 adding 'lm_eval/tasks/multiblimp/multiblimp_pcm.yaml' 2026-05-11T15:53:41,882 adding 'lm_eval/tasks/multiblimp/multiblimp_pol.yaml' 2026-05-11T15:53:41,883 adding 'lm_eval/tasks/multiblimp/multiblimp_por.yaml' 2026-05-11T15:53:41,884 adding 'lm_eval/tasks/multiblimp/multiblimp_quc.yaml' 2026-05-11T15:53:41,886 adding 'lm_eval/tasks/multiblimp/multiblimp_ron.yaml' 2026-05-11T15:53:41,887 adding 'lm_eval/tasks/multiblimp/multiblimp_rus.yaml' 2026-05-11T15:53:41,888 adding 'lm_eval/tasks/multiblimp/multiblimp_sah.yaml' 2026-05-11T15:53:41,889 adding 'lm_eval/tasks/multiblimp/multiblimp_san.yaml' 2026-05-11T15:53:41,890 adding 'lm_eval/tasks/multiblimp/multiblimp_slk.yaml' 2026-05-11T15:53:41,891 adding 'lm_eval/tasks/multiblimp/multiblimp_slv.yaml' 2026-05-11T15:53:41,892 adding 'lm_eval/tasks/multiblimp/multiblimp_sme.yaml' 2026-05-11T15:53:41,893 adding 'lm_eval/tasks/multiblimp/multiblimp_sms.yaml' 2026-05-11T15:53:41,894 adding 'lm_eval/tasks/multiblimp/multiblimp_spa.yaml' 2026-05-11T15:53:41,895 adding 'lm_eval/tasks/multiblimp/multiblimp_sqi.yaml' 2026-05-11T15:53:41,896 adding 'lm_eval/tasks/multiblimp/multiblimp_swe.yaml' 2026-05-11T15:53:41,897 adding 'lm_eval/tasks/multiblimp/multiblimp_tam.yaml' 2026-05-11T15:53:41,898 adding 'lm_eval/tasks/multiblimp/multiblimp_tpn.yaml' 2026-05-11T15:53:41,899 adding 'lm_eval/tasks/multiblimp/multiblimp_ttc.yaml' 2026-05-11T15:53:41,901 adding 'lm_eval/tasks/multiblimp/multiblimp_tur.yaml' 2026-05-11T15:53:41,902 adding 'lm_eval/tasks/multiblimp/multiblimp_uig.yaml' 2026-05-11T15:53:41,903 adding 'lm_eval/tasks/multiblimp/multiblimp_ukr.yaml' 2026-05-11T15:53:41,904 adding 'lm_eval/tasks/multiblimp/multiblimp_urb.yaml' 2026-05-11T15:53:41,905 adding 'lm_eval/tasks/multiblimp/multiblimp_urd.yaml' 2026-05-11T15:53:41,906 adding 'lm_eval/tasks/multiblimp/multiblimp_uzb.yaml' 2026-05-11T15:53:41,907 adding 'lm_eval/tasks/multiblimp/multiblimp_vep.yaml' 2026-05-11T15:53:41,908 adding 'lm_eval/tasks/multiblimp/multiblimp_wbp.yaml' 2026-05-11T15:53:41,909 adding 'lm_eval/tasks/multiblimp/multiblimp_wol.yaml' 2026-05-11T15:53:41,911 adding 'lm_eval/tasks/multiblimp/multiblimp_xcl.yaml' 2026-05-11T15:53:41,912 adding 'lm_eval/tasks/multiblimp/multiblimp_xnr.yaml' 2026-05-11T15:53:41,913 adding 'lm_eval/tasks/multiblimp/multiblimp_xpg.yaml' 2026-05-11T15:53:41,914 adding 'lm_eval/tasks/multiblimp/multiblimp_yrl.yaml' 2026-05-11T15:53:41,916 adding 'lm_eval/tasks/mutual/README.md' 2026-05-11T15:53:41,917 adding 'lm_eval/tasks/mutual/multual_plus.yaml' 2026-05-11T15:53:41,918 adding 'lm_eval/tasks/mutual/mutual.yaml' 2026-05-11T15:53:41,919 adding 'lm_eval/tasks/mutual/utils.py' 2026-05-11T15:53:41,922 adding 'lm_eval/tasks/noreval/README.md' 2026-05-11T15:53:42,002 adding 'lm_eval/tasks/noreval/noreval.jpg' 2026-05-11T15:53:42,008 adding 'lm_eval/tasks/noreval/ask_gec/README.md' 2026-05-11T15:53:42,009 adding 'lm_eval/tasks/noreval/ask_gec/_ask_gec_yaml' 2026-05-11T15:53:42,011 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p0.yaml' 2026-05-11T15:53:42,012 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p1.yaml' 2026-05-11T15:53:42,013 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p2.yaml' 2026-05-11T15:53:42,014 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p3.yaml' 2026-05-11T15:53:42,016 adding 'lm_eval/tasks/noreval/ask_gec/ask_gec_p4.yaml' 2026-05-11T15:53:42,017 adding 'lm_eval/tasks/noreval/ask_gec/errant.py' 2026-05-11T15:53:42,018 adding 'lm_eval/tasks/noreval/ncb/ncb.yaml' 2026-05-11T15:53:42,020 adding 'lm_eval/tasks/noreval/norbelebele/_norbelebele_yaml' 2026-05-11T15:53:42,021 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p0.yaml' 2026-05-11T15:53:42,022 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p1.yaml' 2026-05-11T15:53:42,023 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p2.yaml' 2026-05-11T15:53:42,025 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p3.yaml' 2026-05-11T15:53:42,026 adding 'lm_eval/tasks/noreval/norbelebele/norbelebele_p4.yaml' 2026-05-11T15:53:42,027 adding 'lm_eval/tasks/noreval/norcommonsenseqa/_norcommonsenseqa_yaml' 2026-05-11T15:53:42,029 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p0.yaml' 2026-05-11T15:53:42,030 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p1.yaml' 2026-05-11T15:53:42,032 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p2.yaml' 2026-05-11T15:53:42,033 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p3.yaml' 2026-05-11T15:53:42,034 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nno/norcommonsenseqa_nno_p4.yaml' 2026-05-11T15:53:42,036 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p0.yaml' 2026-05-11T15:53:42,037 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p1.yaml' 2026-05-11T15:53:42,038 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p2.yaml' 2026-05-11T15:53:42,039 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p3.yaml' 2026-05-11T15:53:42,040 adding 'lm_eval/tasks/noreval/norcommonsenseqa/nob/norcommonsenseqa_nob_p4.yaml' 2026-05-11T15:53:42,042 adding 'lm_eval/tasks/noreval/norec/_norec_yaml' 2026-05-11T15:53:42,043 adding 'lm_eval/tasks/noreval/norec/utils.py' 2026-05-11T15:53:42,045 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p0.yaml' 2026-05-11T15:53:42,046 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p1.yaml' 2026-05-11T15:53:42,047 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p2.yaml' 2026-05-11T15:53:42,049 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p3.yaml' 2026-05-11T15:53:42,050 adding 'lm_eval/tasks/noreval/norec/norec_document/norec_document_p4.yaml' 2026-05-11T15:53:42,052 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p0.yaml' 2026-05-11T15:53:42,053 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p1.yaml' 2026-05-11T15:53:42,054 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p2.yaml' 2026-05-11T15:53:42,055 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p3.yaml' 2026-05-11T15:53:42,056 adding 'lm_eval/tasks/noreval/norec/norec_sentence/norec_sentence_p4.yaml' 2026-05-11T15:53:42,058 adding 'lm_eval/tasks/noreval/noridiom/_noridiom_yaml' 2026-05-11T15:53:42,059 adding 'lm_eval/tasks/noreval/noridiom/utils.py' 2026-05-11T15:53:42,061 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p0.yaml' 2026-05-11T15:53:42,062 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p1.yaml' 2026-05-11T15:53:42,064 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p2.yaml' 2026-05-11T15:53:42,065 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p3.yaml' 2026-05-11T15:53:42,066 adding 'lm_eval/tasks/noreval/noridiom/nno/noridiom_nno_p4.yaml' 2026-05-11T15:53:42,068 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p0.yaml' 2026-05-11T15:53:42,069 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p1.yaml' 2026-05-11T15:53:42,070 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p2.yaml' 2026-05-11T15:53:42,072 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p3.yaml' 2026-05-11T15:53:42,073 adding 'lm_eval/tasks/noreval/noridiom/nob/noridiom_nob_p4.yaml' 2026-05-11T15:53:42,075 adding 'lm_eval/tasks/noreval/noropenbookqa/_noropenbookqa_yaml' 2026-05-11T15:53:42,076 adding 'lm_eval/tasks/noreval/noropenbookqa/utils.py' 2026-05-11T15:53:42,078 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p0.yaml' 2026-05-11T15:53:42,079 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p1.yaml' 2026-05-11T15:53:42,080 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p2.yaml' 2026-05-11T15:53:42,081 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p3.yaml' 2026-05-11T15:53:42,082 adding 'lm_eval/tasks/noreval/noropenbookqa/nno/noropenbookqa_nno_p4.yaml' 2026-05-11T15:53:42,084 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p0.yaml' 2026-05-11T15:53:42,085 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p1.yaml' 2026-05-11T15:53:42,086 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p2.yaml' 2026-05-11T15:53:42,087 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p3.yaml' 2026-05-11T15:53:42,088 adding 'lm_eval/tasks/noreval/noropenbookqa/nob/noropenbookqa_nob_p4.yaml' 2026-05-11T15:53:42,090 adding 'lm_eval/tasks/noreval/norquad/_norquad_yaml' 2026-05-11T15:53:42,092 adding 'lm_eval/tasks/noreval/norquad/norquad_p0.yaml' 2026-05-11T15:53:42,093 adding 'lm_eval/tasks/noreval/norquad/norquad_p1.yaml' 2026-05-11T15:53:42,094 adding 'lm_eval/tasks/noreval/norquad/norquad_p2.yaml' 2026-05-11T15:53:42,095 adding 'lm_eval/tasks/noreval/norquad/norquad_p3.yaml' 2026-05-11T15:53:42,096 adding 'lm_eval/tasks/noreval/norquad/norquad_p4.yaml' 2026-05-11T15:53:42,097 adding 'lm_eval/tasks/noreval/norquad/utils.py' 2026-05-11T15:53:42,099 adding 'lm_eval/tasks/noreval/norrewrite-instruct/norrewrite_instruct.yaml' 2026-05-11T15:53:42,101 adding 'lm_eval/tasks/noreval/norsumm/_norsumm_yaml' 2026-05-11T15:53:42,102 adding 'lm_eval/tasks/noreval/norsumm/utils.py' 2026-05-11T15:53:42,104 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p0.yaml' 2026-05-11T15:53:42,105 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p1.yaml' 2026-05-11T15:53:42,107 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p2.yaml' 2026-05-11T15:53:42,108 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p3.yaml' 2026-05-11T15:53:42,109 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p4.yaml' 2026-05-11T15:53:42,110 adding 'lm_eval/tasks/noreval/norsumm/nno/norsumm_nno_p5.yaml' 2026-05-11T15:53:42,112 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p0.yaml' 2026-05-11T15:53:42,113 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p1.yaml' 2026-05-11T15:53:42,114 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p2.yaml' 2026-05-11T15:53:42,115 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p3.yaml' 2026-05-11T15:53:42,117 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p4.yaml' 2026-05-11T15:53:42,118 adding 'lm_eval/tasks/noreval/norsumm/nob/norsumm_nob_p5.yaml' 2026-05-11T15:53:42,120 adding 'lm_eval/tasks/noreval/norsummarize-instruct/norsummarize_instruct.yaml' 2026-05-11T15:53:42,122 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/_nortruthfulqa_gen_yaml' 2026-05-11T15:53:42,123 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/utils.py' 2026-05-11T15:53:42,125 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p0.yaml' 2026-05-11T15:53:42,126 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p1.yaml' 2026-05-11T15:53:42,128 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p2.yaml' 2026-05-11T15:53:42,129 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p3.yaml' 2026-05-11T15:53:42,130 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nno/nortruthfulqa_gen_nno_p4.yaml' 2026-05-11T15:53:42,132 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p0.yaml' 2026-05-11T15:53:42,133 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p1.yaml' 2026-05-11T15:53:42,134 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p2.yaml' 2026-05-11T15:53:42,135 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p3.yaml' 2026-05-11T15:53:42,136 adding 'lm_eval/tasks/noreval/nortruthfulqa/generation/nob/nortruthfulqa_gen_nob_p4.yaml' 2026-05-11T15:53:42,138 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/_nortruthfulqa_mc_yaml' 2026-05-11T15:53:42,140 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p0.yaml' 2026-05-11T15:53:42,141 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p1.yaml' 2026-05-11T15:53:42,142 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p2.yaml' 2026-05-11T15:53:42,143 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p3.yaml' 2026-05-11T15:53:42,144 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/nortruthfulqa_mc_nno_p4.yaml' 2026-05-11T15:53:42,145 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nno/utils.py' 2026-05-11T15:53:42,147 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p0.yaml' 2026-05-11T15:53:42,148 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p1.yaml' 2026-05-11T15:53:42,149 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p2.yaml' 2026-05-11T15:53:42,150 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p3.yaml' 2026-05-11T15:53:42,152 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/nortruthfulqa_mc_nob_p4.yaml' 2026-05-11T15:53:42,153 adding 'lm_eval/tasks/noreval/nortruthfulqa/multiple_choice/nob/utils.py' 2026-05-11T15:53:42,154 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/_nrk_quiz_qa_yaml' 2026-05-11T15:53:42,156 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p0.yaml' 2026-05-11T15:53:42,157 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p1.yaml' 2026-05-11T15:53:42,158 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p2.yaml' 2026-05-11T15:53:42,159 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p3.yaml' 2026-05-11T15:53:42,160 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/nrk_quiz_qa_nno_p4.yaml' 2026-05-11T15:53:42,162 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nno/utils.py' 2026-05-11T15:53:42,163 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p0.yaml' 2026-05-11T15:53:42,165 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p1.yaml' 2026-05-11T15:53:42,166 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p2.yaml' 2026-05-11T15:53:42,167 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p3.yaml' 2026-05-11T15:53:42,168 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/nrk_quiz_qa_nob_p4.yaml' 2026-05-11T15:53:42,169 adding 'lm_eval/tasks/noreval/nrk_quiz_qa/nob/utils.py' 2026-05-11T15:53:42,171 adding 'lm_eval/tasks/noreval/tatoeba/_tatoeba_yaml' 2026-05-11T15:53:42,173 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p0.yaml' 2026-05-11T15:53:42,174 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p1.yaml' 2026-05-11T15:53:42,175 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p2.yaml' 2026-05-11T15:53:42,176 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nno/tatoeba_eng_nno_p3.yaml' 2026-05-11T15:53:42,178 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p0.yaml' 2026-05-11T15:53:42,179 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p1.yaml' 2026-05-11T15:53:42,180 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p2.yaml' 2026-05-11T15:53:42,181 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_eng_nob/tatoeba_eng_nob_p3.yaml' 2026-05-11T15:53:42,183 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p0.yaml' 2026-05-11T15:53:42,184 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p1.yaml' 2026-05-11T15:53:42,185 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p2.yaml' 2026-05-11T15:53:42,186 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nno_eng/tatoeba_nno_eng_p3.yaml' 2026-05-11T15:53:42,188 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p0.yaml' 2026-05-11T15:53:42,189 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p1.yaml' 2026-05-11T15:53:42,190 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p2.yaml' 2026-05-11T15:53:42,191 adding 'lm_eval/tasks/noreval/tatoeba/tatoeba_nob_eng/tatoeba_nob_eng_p3.yaml' 2026-05-11T15:53:42,193 adding 'lm_eval/tasks/noticia/README.md' 2026-05-11T15:53:42,195 adding 'lm_eval/tasks/noticia/noticia.yaml' 2026-05-11T15:53:42,196 adding 'lm_eval/tasks/noticia/utils.py' 2026-05-11T15:53:42,197 adding 'lm_eval/tasks/nq_open/README.md' 2026-05-11T15:53:42,199 adding 'lm_eval/tasks/nq_open/nq_open.yaml' 2026-05-11T15:53:42,201 adding 'lm_eval/tasks/okapi/arc_multilingual/README.md' 2026-05-11T15:53:42,203 adding 'lm_eval/tasks/okapi/arc_multilingual/_arc_yaml' 2026-05-11T15:53:42,204 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ar.yaml' 2026-05-11T15:53:42,205 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_bn.yaml' 2026-05-11T15:53:42,206 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ca.yaml' 2026-05-11T15:53:42,207 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_da.yaml' 2026-05-11T15:53:42,208 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_de.yaml' 2026-05-11T15:53:42,209 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_es.yaml' 2026-05-11T15:53:42,210 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_eu.yaml' 2026-05-11T15:53:42,211 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_fr.yaml' 2026-05-11T15:53:42,212 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_gu.yaml' 2026-05-11T15:53:42,214 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hi.yaml' 2026-05-11T15:53:42,215 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hr.yaml' 2026-05-11T15:53:42,216 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hu.yaml' 2026-05-11T15:53:42,217 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_hy.yaml' 2026-05-11T15:53:42,218 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_id.yaml' 2026-05-11T15:53:42,219 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_it.yaml' 2026-05-11T15:53:42,220 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_kn.yaml' 2026-05-11T15:53:42,222 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ml.yaml' 2026-05-11T15:53:42,223 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_mr.yaml' 2026-05-11T15:53:42,224 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ne.yaml' 2026-05-11T15:53:42,225 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_nl.yaml' 2026-05-11T15:53:42,226 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_pt.yaml' 2026-05-11T15:53:42,227 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ro.yaml' 2026-05-11T15:53:42,229 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ru.yaml' 2026-05-11T15:53:42,230 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sk.yaml' 2026-05-11T15:53:42,231 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sr.yaml' 2026-05-11T15:53:42,232 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_sv.yaml' 2026-05-11T15:53:42,233 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_ta.yaml' 2026-05-11T15:53:42,234 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_te.yaml' 2026-05-11T15:53:42,235 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_uk.yaml' 2026-05-11T15:53:42,237 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_vi.yaml' 2026-05-11T15:53:42,238 adding 'lm_eval/tasks/okapi/arc_multilingual/arc_zh.yaml' 2026-05-11T15:53:42,239 adding 'lm_eval/tasks/okapi/arc_multilingual/utils.py' 2026-05-11T15:53:42,242 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/README.md' 2026-05-11T15:53:42,243 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/_hellaswag_yaml' 2026-05-11T15:53:42,244 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ar.yaml' 2026-05-11T15:53:42,245 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_bn.yaml' 2026-05-11T15:53:42,246 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ca.yaml' 2026-05-11T15:53:42,247 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_da.yaml' 2026-05-11T15:53:42,248 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_de.yaml' 2026-05-11T15:53:42,250 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_es.yaml' 2026-05-11T15:53:42,251 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_eu.yaml' 2026-05-11T15:53:42,252 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_fr.yaml' 2026-05-11T15:53:42,253 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_gu.yaml' 2026-05-11T15:53:42,254 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hi.yaml' 2026-05-11T15:53:42,255 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hr.yaml' 2026-05-11T15:53:42,256 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hu.yaml' 2026-05-11T15:53:42,257 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_hy.yaml' 2026-05-11T15:53:42,259 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_id.yaml' 2026-05-11T15:53:42,260 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_it.yaml' 2026-05-11T15:53:42,261 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_kn.yaml' 2026-05-11T15:53:42,262 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ml.yaml' 2026-05-11T15:53:42,263 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_mr.yaml' 2026-05-11T15:53:42,264 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ne.yaml' 2026-05-11T15:53:42,265 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_nl.yaml' 2026-05-11T15:53:42,266 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_pt.yaml' 2026-05-11T15:53:42,267 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ro.yaml' 2026-05-11T15:53:42,269 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ru.yaml' 2026-05-11T15:53:42,270 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sk.yaml' 2026-05-11T15:53:42,271 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sr.yaml' 2026-05-11T15:53:42,272 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_sv.yaml' 2026-05-11T15:53:42,273 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_ta.yaml' 2026-05-11T15:53:42,274 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_te.yaml' 2026-05-11T15:53:42,276 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_uk.yaml' 2026-05-11T15:53:42,277 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/hellaswag_vi.yaml' 2026-05-11T15:53:42,278 adding 'lm_eval/tasks/okapi/hellaswag_multilingual/utils.py' 2026-05-11T15:53:42,280 adding 'lm_eval/tasks/okapi/mmlu_multilingual/_default_yaml' 2026-05-11T15:53:42,281 adding 'lm_eval/tasks/okapi/mmlu_multilingual/_generate_configs.py' 2026-05-11T15:53:42,283 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ar.yaml' 2026-05-11T15:53:42,284 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_bn.yaml' 2026-05-11T15:53:42,285 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ca.yaml' 2026-05-11T15:53:42,286 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_da.yaml' 2026-05-11T15:53:42,287 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_de.yaml' 2026-05-11T15:53:42,288 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_en.yaml' 2026-05-11T15:53:42,289 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_es.yaml' 2026-05-11T15:53:42,291 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_eu.yaml' 2026-05-11T15:53:42,292 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_fr.yaml' 2026-05-11T15:53:42,293 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_gu.yaml' 2026-05-11T15:53:42,294 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hi.yaml' 2026-05-11T15:53:42,295 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hr.yaml' 2026-05-11T15:53:42,296 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hu.yaml' 2026-05-11T15:53:42,297 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_hy.yaml' 2026-05-11T15:53:42,298 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_id.yaml' 2026-05-11T15:53:42,299 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_is.yaml' 2026-05-11T15:53:42,300 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_it.yaml' 2026-05-11T15:53:42,301 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_kn.yaml' 2026-05-11T15:53:42,302 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ml.yaml' 2026-05-11T15:53:42,303 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml' 2026-05-11T15:53:42,305 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml' 2026-05-11T15:53:42,306 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml' 2026-05-11T15:53:42,307 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml' 2026-05-11T15:53:42,308 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml' 2026-05-11T15:53:42,309 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml' 2026-05-11T15:53:42,311 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml' 2026-05-11T15:53:42,312 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sk.yaml' 2026-05-11T15:53:42,313 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml' 2026-05-11T15:53:42,314 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml' 2026-05-11T15:53:42,315 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml' 2026-05-11T15:53:42,316 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml' 2026-05-11T15:53:42,317 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml' 2026-05-11T15:53:42,319 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml' 2026-05-11T15:53:42,320 adding 'lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml' 2026-05-11T15:53:42,323 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/README.md' 2026-05-11T15:53:42,324 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc1_yaml' 2026-05-11T15:53:42,326 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/_truthfulqa_mc2_yaml' 2026-05-11T15:53:42,327 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc1.yaml' 2026-05-11T15:53:42,328 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ar_mc2.yaml' 2026-05-11T15:53:42,329 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc1.yaml' 2026-05-11T15:53:42,330 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_bn_mc2.yaml' 2026-05-11T15:53:42,332 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc1.yaml' 2026-05-11T15:53:42,333 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ca_mc2.yaml' 2026-05-11T15:53:42,334 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc1.yaml' 2026-05-11T15:53:42,335 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_da_mc2.yaml' 2026-05-11T15:53:42,336 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc1.yaml' 2026-05-11T15:53:42,337 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_de_mc2.yaml' 2026-05-11T15:53:42,339 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc1.yaml' 2026-05-11T15:53:42,340 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_es_mc2.yaml' 2026-05-11T15:53:42,341 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc1.yaml' 2026-05-11T15:53:42,342 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_eu_mc2.yaml' 2026-05-11T15:53:42,344 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc1.yaml' 2026-05-11T15:53:42,345 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_fr_mc2.yaml' 2026-05-11T15:53:42,346 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc1.yaml' 2026-05-11T15:53:42,347 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_gu_mc2.yaml' 2026-05-11T15:53:42,348 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc1.yaml' 2026-05-11T15:53:42,349 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hi_mc2.yaml' 2026-05-11T15:53:42,350 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc1.yaml' 2026-05-11T15:53:42,351 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hr_mc2.yaml' 2026-05-11T15:53:42,353 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc1.yaml' 2026-05-11T15:53:42,354 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hu_mc2.yaml' 2026-05-11T15:53:42,355 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc1.yaml' 2026-05-11T15:53:42,356 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_hy_mc2.yaml' 2026-05-11T15:53:42,357 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc1.yaml' 2026-05-11T15:53:42,358 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_id_mc2.yaml' 2026-05-11T15:53:42,359 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc1.yaml' 2026-05-11T15:53:42,361 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_it_mc2.yaml' 2026-05-11T15:53:42,362 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc1.yaml' 2026-05-11T15:53:42,363 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_kn_mc2.yaml' 2026-05-11T15:53:42,364 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc1.yaml' 2026-05-11T15:53:42,365 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ml_mc2.yaml' 2026-05-11T15:53:42,366 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc1.yaml' 2026-05-11T15:53:42,367 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_mr_mc2.yaml' 2026-05-11T15:53:42,369 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc1.yaml' 2026-05-11T15:53:42,370 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ne_mc2.yaml' 2026-05-11T15:53:42,371 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc1.yaml' 2026-05-11T15:53:42,372 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_nl_mc2.yaml' 2026-05-11T15:53:42,373 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc1.yaml' 2026-05-11T15:53:42,374 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_pt_mc2.yaml' 2026-05-11T15:53:42,376 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc1.yaml' 2026-05-11T15:53:42,377 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ro_mc2.yaml' 2026-05-11T15:53:42,378 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc1.yaml' 2026-05-11T15:53:42,380 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ru_mc2.yaml' 2026-05-11T15:53:42,381 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc1.yaml' 2026-05-11T15:53:42,382 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sk_mc2.yaml' 2026-05-11T15:53:42,383 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc1.yaml' 2026-05-11T15:53:42,384 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sr_mc2.yaml' 2026-05-11T15:53:42,386 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc1.yaml' 2026-05-11T15:53:42,387 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_sv_mc2.yaml' 2026-05-11T15:53:42,388 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc1.yaml' 2026-05-11T15:53:42,389 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_ta_mc2.yaml' 2026-05-11T15:53:42,390 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc1.yaml' 2026-05-11T15:53:42,392 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_te_mc2.yaml' 2026-05-11T15:53:42,393 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc1.yaml' 2026-05-11T15:53:42,394 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_uk_mc2.yaml' 2026-05-11T15:53:42,395 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc1.yaml' 2026-05-11T15:53:42,396 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_vi_mc2.yaml' 2026-05-11T15:53:42,397 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc1.yaml' 2026-05-11T15:53:42,399 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/truthfulqa_zh_mc2.yaml' 2026-05-11T15:53:42,400 adding 'lm_eval/tasks/okapi/truthfulqa_multilingual/utils.py' 2026-05-11T15:53:42,402 adding 'lm_eval/tasks/olaph/README.md' 2026-05-11T15:53:42,403 adding 'lm_eval/tasks/olaph/olaph.yaml' 2026-05-11T15:53:42,404 adding 'lm_eval/tasks/olaph/olaph_perplexity.yaml' 2026-05-11T15:53:42,406 adding 'lm_eval/tasks/olaph/utils.py' 2026-05-11T15:53:42,407 adding 'lm_eval/tasks/olaph/utils_perplexity.py' 2026-05-11T15:53:42,409 adding 'lm_eval/tasks/openai-mmmlu/README.md' 2026-05-11T15:53:42,411 adding 'lm_eval/tasks/openai-mmmlu/_generate_configs.py' 2026-05-11T15:53:42,412 adding 'lm_eval/tasks/openai-mmmlu/languages.json' 2026-05-11T15:53:42,413 adding 'lm_eval/tasks/openai-mmmlu/subjects.json' 2026-05-11T15:53:42,472 adding 'lm_eval/tasks/openai-mmmlu/default/_default_template_yaml' 2026-05-11T15:53:42,473 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu.yaml' 2026-05-11T15:53:42,474 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy.yaml' 2026-05-11T15:53:42,475 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_humanities.yaml' 2026-05-11T15:53:42,476 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_other.yaml' 2026-05-11T15:53:42,478 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_social_sciences.yaml' 2026-05-11T15:53:42,479 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ar_xy_stem.yaml' 2026-05-11T15:53:42,480 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd.yaml' 2026-05-11T15:53:42,481 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_humanities.yaml' 2026-05-11T15:53:42,483 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_other.yaml' 2026-05-11T15:53:42,484 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_social_sciences.yaml' 2026-05-11T15:53:42,485 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_bn_bd_stem.yaml' 2026-05-11T15:53:42,486 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de.yaml' 2026-05-11T15:53:42,488 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_humanities.yaml' 2026-05-11T15:53:42,489 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_other.yaml' 2026-05-11T15:53:42,490 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_social_sciences.yaml' 2026-05-11T15:53:42,491 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_de_de_stem.yaml' 2026-05-11T15:53:42,492 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la.yaml' 2026-05-11T15:53:42,494 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_humanities.yaml' 2026-05-11T15:53:42,495 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_other.yaml' 2026-05-11T15:53:42,496 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_social_sciences.yaml' 2026-05-11T15:53:42,497 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_es_la_stem.yaml' 2026-05-11T15:53:42,499 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr.yaml' 2026-05-11T15:53:42,500 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_humanities.yaml' 2026-05-11T15:53:42,501 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_other.yaml' 2026-05-11T15:53:42,502 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_social_sciences.yaml' 2026-05-11T15:53:42,503 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_fr_fr_stem.yaml' 2026-05-11T15:53:42,505 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in.yaml' 2026-05-11T15:53:42,506 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_humanities.yaml' 2026-05-11T15:53:42,507 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_other.yaml' 2026-05-11T15:53:42,508 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_social_sciences.yaml' 2026-05-11T15:53:42,509 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_hi_in_stem.yaml' 2026-05-11T15:53:42,511 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id.yaml' 2026-05-11T15:53:42,512 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_humanities.yaml' 2026-05-11T15:53:42,513 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_other.yaml' 2026-05-11T15:53:42,514 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_social_sciences.yaml' 2026-05-11T15:53:42,515 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_id_id_stem.yaml' 2026-05-11T15:53:42,516 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it.yaml' 2026-05-11T15:53:42,517 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_humanities.yaml' 2026-05-11T15:53:42,519 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_other.yaml' 2026-05-11T15:53:42,520 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_social_sciences.yaml' 2026-05-11T15:53:42,521 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_it_it_stem.yaml' 2026-05-11T15:53:42,522 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp.yaml' 2026-05-11T15:53:42,523 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_humanities.yaml' 2026-05-11T15:53:42,524 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_other.yaml' 2026-05-11T15:53:42,526 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_social_sciences.yaml' 2026-05-11T15:53:42,527 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ja_jp_stem.yaml' 2026-05-11T15:53:42,528 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr.yaml' 2026-05-11T15:53:42,529 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_humanities.yaml' 2026-05-11T15:53:42,531 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_other.yaml' 2026-05-11T15:53:42,532 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_social_sciences.yaml' 2026-05-11T15:53:42,533 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_ko_kr_stem.yaml' 2026-05-11T15:53:42,534 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br.yaml' 2026-05-11T15:53:42,535 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_humanities.yaml' 2026-05-11T15:53:42,537 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_other.yaml' 2026-05-11T15:53:42,538 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_social_sciences.yaml' 2026-05-11T15:53:42,539 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_pt_br_stem.yaml' 2026-05-11T15:53:42,540 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke.yaml' 2026-05-11T15:53:42,541 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_humanities.yaml' 2026-05-11T15:53:42,543 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_other.yaml' 2026-05-11T15:53:42,544 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_social_sciences.yaml' 2026-05-11T15:53:42,545 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_sw_ke_stem.yaml' 2026-05-11T15:53:42,546 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng.yaml' 2026-05-11T15:53:42,547 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_humanities.yaml' 2026-05-11T15:53:42,549 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_other.yaml' 2026-05-11T15:53:42,550 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_social_sciences.yaml' 2026-05-11T15:53:42,551 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_yo_ng_stem.yaml' 2026-05-11T15:53:42,553 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn.yaml' 2026-05-11T15:53:42,554 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_humanities.yaml' 2026-05-11T15:53:42,555 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_other.yaml' 2026-05-11T15:53:42,556 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_social_sciences.yaml' 2026-05-11T15:53:42,557 adding 'lm_eval/tasks/openai-mmmlu/default/_mmmlu_zh_cn_stem.yaml' 2026-05-11T15:53:42,559 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_abstract_algebra.yaml' 2026-05-11T15:53:42,560 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_anatomy.yaml' 2026-05-11T15:53:42,561 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_astronomy.yaml' 2026-05-11T15:53:42,562 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_business_ethics.yaml' 2026-05-11T15:53:42,564 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_clinical_knowledge.yaml' 2026-05-11T15:53:42,565 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_biology.yaml' 2026-05-11T15:53:42,566 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_chemistry.yaml' 2026-05-11T15:53:42,567 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_computer_science.yaml' 2026-05-11T15:53:42,568 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_mathematics.yaml' 2026-05-11T15:53:42,569 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_medicine.yaml' 2026-05-11T15:53:42,571 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_college_physics.yaml' 2026-05-11T15:53:42,572 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_computer_security.yaml' 2026-05-11T15:53:42,573 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_conceptual_physics.yaml' 2026-05-11T15:53:42,574 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_econometrics.yaml' 2026-05-11T15:53:42,575 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_electrical_engineering.yaml' 2026-05-11T15:53:42,577 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_elementary_mathematics.yaml' 2026-05-11T15:53:42,578 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_formal_logic.yaml' 2026-05-11T15:53:42,579 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_global_facts.yaml' 2026-05-11T15:53:42,580 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_biology.yaml' 2026-05-11T15:53:42,581 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_chemistry.yaml' 2026-05-11T15:53:42,583 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_computer_science.yaml' 2026-05-11T15:53:42,584 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_european_history.yaml' 2026-05-11T15:53:42,585 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_geography.yaml' 2026-05-11T15:53:42,586 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,587 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_macroeconomics.yaml' 2026-05-11T15:53:42,589 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_mathematics.yaml' 2026-05-11T15:53:42,590 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_microeconomics.yaml' 2026-05-11T15:53:42,591 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_physics.yaml' 2026-05-11T15:53:42,592 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_psychology.yaml' 2026-05-11T15:53:42,594 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_statistics.yaml' 2026-05-11T15:53:42,595 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_us_history.yaml' 2026-05-11T15:53:42,596 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_high_school_world_history.yaml' 2026-05-11T15:53:42,597 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_aging.yaml' 2026-05-11T15:53:42,598 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_human_sexuality.yaml' 2026-05-11T15:53:42,600 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_international_law.yaml' 2026-05-11T15:53:42,601 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_jurisprudence.yaml' 2026-05-11T15:53:42,602 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_logical_fallacies.yaml' 2026-05-11T15:53:42,603 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_machine_learning.yaml' 2026-05-11T15:53:42,605 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_management.yaml' 2026-05-11T15:53:42,606 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_marketing.yaml' 2026-05-11T15:53:42,607 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_medical_genetics.yaml' 2026-05-11T15:53:42,608 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_miscellaneous.yaml' 2026-05-11T15:53:42,609 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_disputes.yaml' 2026-05-11T15:53:42,610 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_moral_scenarios.yaml' 2026-05-11T15:53:42,611 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_nutrition.yaml' 2026-05-11T15:53:42,612 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_philosophy.yaml' 2026-05-11T15:53:42,614 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_prehistory.yaml' 2026-05-11T15:53:42,615 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_accounting.yaml' 2026-05-11T15:53:42,616 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_law.yaml' 2026-05-11T15:53:42,617 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_medicine.yaml' 2026-05-11T15:53:42,618 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_professional_psychology.yaml' 2026-05-11T15:53:42,619 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_public_relations.yaml' 2026-05-11T15:53:42,621 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_security_studies.yaml' 2026-05-11T15:53:42,622 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_sociology.yaml' 2026-05-11T15:53:42,623 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_us_foreign_policy.yaml' 2026-05-11T15:53:42,624 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_virology.yaml' 2026-05-11T15:53:42,625 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ar_xy_world_religions.yaml' 2026-05-11T15:53:42,627 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_abstract_algebra.yaml' 2026-05-11T15:53:42,628 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_anatomy.yaml' 2026-05-11T15:53:42,629 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_astronomy.yaml' 2026-05-11T15:53:42,630 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_business_ethics.yaml' 2026-05-11T15:53:42,631 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_clinical_knowledge.yaml' 2026-05-11T15:53:42,633 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_biology.yaml' 2026-05-11T15:53:42,634 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_chemistry.yaml' 2026-05-11T15:53:42,635 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_computer_science.yaml' 2026-05-11T15:53:42,636 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_mathematics.yaml' 2026-05-11T15:53:42,637 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_medicine.yaml' 2026-05-11T15:53:42,639 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_college_physics.yaml' 2026-05-11T15:53:42,640 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_computer_security.yaml' 2026-05-11T15:53:42,641 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_conceptual_physics.yaml' 2026-05-11T15:53:42,642 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_econometrics.yaml' 2026-05-11T15:53:42,644 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_electrical_engineering.yaml' 2026-05-11T15:53:42,645 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_elementary_mathematics.yaml' 2026-05-11T15:53:42,646 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_formal_logic.yaml' 2026-05-11T15:53:42,647 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_global_facts.yaml' 2026-05-11T15:53:42,649 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_biology.yaml' 2026-05-11T15:53:42,650 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_chemistry.yaml' 2026-05-11T15:53:42,651 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_computer_science.yaml' 2026-05-11T15:53:42,653 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_european_history.yaml' 2026-05-11T15:53:42,654 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_geography.yaml' 2026-05-11T15:53:42,655 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,656 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_macroeconomics.yaml' 2026-05-11T15:53:42,657 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_mathematics.yaml' 2026-05-11T15:53:42,658 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_microeconomics.yaml' 2026-05-11T15:53:42,660 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_physics.yaml' 2026-05-11T15:53:42,661 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_psychology.yaml' 2026-05-11T15:53:42,662 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_statistics.yaml' 2026-05-11T15:53:42,663 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_us_history.yaml' 2026-05-11T15:53:42,664 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_high_school_world_history.yaml' 2026-05-11T15:53:42,666 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_aging.yaml' 2026-05-11T15:53:42,667 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_human_sexuality.yaml' 2026-05-11T15:53:42,668 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_international_law.yaml' 2026-05-11T15:53:42,669 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_jurisprudence.yaml' 2026-05-11T15:53:42,670 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_logical_fallacies.yaml' 2026-05-11T15:53:42,671 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_machine_learning.yaml' 2026-05-11T15:53:42,672 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_management.yaml' 2026-05-11T15:53:42,674 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_marketing.yaml' 2026-05-11T15:53:42,675 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_medical_genetics.yaml' 2026-05-11T15:53:42,676 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_miscellaneous.yaml' 2026-05-11T15:53:42,677 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_disputes.yaml' 2026-05-11T15:53:42,678 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_moral_scenarios.yaml' 2026-05-11T15:53:42,680 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_nutrition.yaml' 2026-05-11T15:53:42,681 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_philosophy.yaml' 2026-05-11T15:53:42,682 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_prehistory.yaml' 2026-05-11T15:53:42,683 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_accounting.yaml' 2026-05-11T15:53:42,685 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_law.yaml' 2026-05-11T15:53:42,686 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_medicine.yaml' 2026-05-11T15:53:42,687 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_professional_psychology.yaml' 2026-05-11T15:53:42,689 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_public_relations.yaml' 2026-05-11T15:53:42,690 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_security_studies.yaml' 2026-05-11T15:53:42,691 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_sociology.yaml' 2026-05-11T15:53:42,692 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_us_foreign_policy.yaml' 2026-05-11T15:53:42,693 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_virology.yaml' 2026-05-11T15:53:42,695 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_bn_bd_world_religions.yaml' 2026-05-11T15:53:42,696 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_abstract_algebra.yaml' 2026-05-11T15:53:42,697 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_anatomy.yaml' 2026-05-11T15:53:42,698 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_astronomy.yaml' 2026-05-11T15:53:42,700 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_business_ethics.yaml' 2026-05-11T15:53:42,701 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_clinical_knowledge.yaml' 2026-05-11T15:53:42,702 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_biology.yaml' 2026-05-11T15:53:42,703 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_chemistry.yaml' 2026-05-11T15:53:42,705 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_computer_science.yaml' 2026-05-11T15:53:42,706 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_mathematics.yaml' 2026-05-11T15:53:42,707 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_medicine.yaml' 2026-05-11T15:53:42,708 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_college_physics.yaml' 2026-05-11T15:53:42,709 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_computer_security.yaml' 2026-05-11T15:53:42,710 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_conceptual_physics.yaml' 2026-05-11T15:53:42,712 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_econometrics.yaml' 2026-05-11T15:53:42,713 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_electrical_engineering.yaml' 2026-05-11T15:53:42,714 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_elementary_mathematics.yaml' 2026-05-11T15:53:42,715 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_formal_logic.yaml' 2026-05-11T15:53:42,717 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_global_facts.yaml' 2026-05-11T15:53:42,718 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_biology.yaml' 2026-05-11T15:53:42,719 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_chemistry.yaml' 2026-05-11T15:53:42,720 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_computer_science.yaml' 2026-05-11T15:53:42,721 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_european_history.yaml' 2026-05-11T15:53:42,723 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_geography.yaml' 2026-05-11T15:53:42,724 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,725 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_macroeconomics.yaml' 2026-05-11T15:53:42,726 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_mathematics.yaml' 2026-05-11T15:53:42,727 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_microeconomics.yaml' 2026-05-11T15:53:42,729 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_physics.yaml' 2026-05-11T15:53:42,730 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_psychology.yaml' 2026-05-11T15:53:42,731 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_statistics.yaml' 2026-05-11T15:53:42,732 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_us_history.yaml' 2026-05-11T15:53:42,733 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_high_school_world_history.yaml' 2026-05-11T15:53:42,735 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_aging.yaml' 2026-05-11T15:53:42,736 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_human_sexuality.yaml' 2026-05-11T15:53:42,738 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_international_law.yaml' 2026-05-11T15:53:42,739 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_jurisprudence.yaml' 2026-05-11T15:53:42,740 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_logical_fallacies.yaml' 2026-05-11T15:53:42,741 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_machine_learning.yaml' 2026-05-11T15:53:42,743 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_management.yaml' 2026-05-11T15:53:42,744 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_marketing.yaml' 2026-05-11T15:53:42,745 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_medical_genetics.yaml' 2026-05-11T15:53:42,746 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_miscellaneous.yaml' 2026-05-11T15:53:42,747 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_disputes.yaml' 2026-05-11T15:53:42,749 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_moral_scenarios.yaml' 2026-05-11T15:53:42,750 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_nutrition.yaml' 2026-05-11T15:53:42,751 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_philosophy.yaml' 2026-05-11T15:53:42,752 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_prehistory.yaml' 2026-05-11T15:53:42,753 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_accounting.yaml' 2026-05-11T15:53:42,754 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_law.yaml' 2026-05-11T15:53:42,756 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_medicine.yaml' 2026-05-11T15:53:42,757 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_professional_psychology.yaml' 2026-05-11T15:53:42,758 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_public_relations.yaml' 2026-05-11T15:53:42,759 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_security_studies.yaml' 2026-05-11T15:53:42,761 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_sociology.yaml' 2026-05-11T15:53:42,762 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_us_foreign_policy.yaml' 2026-05-11T15:53:42,763 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_virology.yaml' 2026-05-11T15:53:42,764 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_de_de_world_religions.yaml' 2026-05-11T15:53:42,765 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_abstract_algebra.yaml' 2026-05-11T15:53:42,766 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_anatomy.yaml' 2026-05-11T15:53:42,768 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_astronomy.yaml' 2026-05-11T15:53:42,769 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_business_ethics.yaml' 2026-05-11T15:53:42,770 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_clinical_knowledge.yaml' 2026-05-11T15:53:42,771 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_biology.yaml' 2026-05-11T15:53:42,772 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_chemistry.yaml' 2026-05-11T15:53:42,773 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_computer_science.yaml' 2026-05-11T15:53:42,775 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_mathematics.yaml' 2026-05-11T15:53:42,776 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_medicine.yaml' 2026-05-11T15:53:42,777 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_college_physics.yaml' 2026-05-11T15:53:42,778 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_computer_security.yaml' 2026-05-11T15:53:42,779 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_conceptual_physics.yaml' 2026-05-11T15:53:42,780 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_econometrics.yaml' 2026-05-11T15:53:42,782 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_electrical_engineering.yaml' 2026-05-11T15:53:42,783 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_elementary_mathematics.yaml' 2026-05-11T15:53:42,784 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_formal_logic.yaml' 2026-05-11T15:53:42,785 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_global_facts.yaml' 2026-05-11T15:53:42,787 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_biology.yaml' 2026-05-11T15:53:42,788 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_chemistry.yaml' 2026-05-11T15:53:42,789 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_computer_science.yaml' 2026-05-11T15:53:42,790 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_european_history.yaml' 2026-05-11T15:53:42,792 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_geography.yaml' 2026-05-11T15:53:42,793 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,794 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_macroeconomics.yaml' 2026-05-11T15:53:42,795 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_mathematics.yaml' 2026-05-11T15:53:42,796 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_microeconomics.yaml' 2026-05-11T15:53:42,797 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_physics.yaml' 2026-05-11T15:53:42,799 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_psychology.yaml' 2026-05-11T15:53:42,800 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_statistics.yaml' 2026-05-11T15:53:42,801 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_us_history.yaml' 2026-05-11T15:53:42,802 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_high_school_world_history.yaml' 2026-05-11T15:53:42,803 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_aging.yaml' 2026-05-11T15:53:42,805 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_human_sexuality.yaml' 2026-05-11T15:53:42,806 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_international_law.yaml' 2026-05-11T15:53:42,807 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_jurisprudence.yaml' 2026-05-11T15:53:42,808 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_logical_fallacies.yaml' 2026-05-11T15:53:42,810 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_machine_learning.yaml' 2026-05-11T15:53:42,811 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_management.yaml' 2026-05-11T15:53:42,812 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_marketing.yaml' 2026-05-11T15:53:42,813 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_medical_genetics.yaml' 2026-05-11T15:53:42,814 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_miscellaneous.yaml' 2026-05-11T15:53:42,816 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_disputes.yaml' 2026-05-11T15:53:42,817 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_moral_scenarios.yaml' 2026-05-11T15:53:42,818 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_nutrition.yaml' 2026-05-11T15:53:42,819 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_philosophy.yaml' 2026-05-11T15:53:42,820 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_prehistory.yaml' 2026-05-11T15:53:42,822 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_accounting.yaml' 2026-05-11T15:53:42,823 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_law.yaml' 2026-05-11T15:53:42,824 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_medicine.yaml' 2026-05-11T15:53:42,826 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_professional_psychology.yaml' 2026-05-11T15:53:42,827 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_public_relations.yaml' 2026-05-11T15:53:42,828 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_security_studies.yaml' 2026-05-11T15:53:42,829 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_sociology.yaml' 2026-05-11T15:53:42,830 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_us_foreign_policy.yaml' 2026-05-11T15:53:42,832 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_virology.yaml' 2026-05-11T15:53:42,833 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_es_la_world_religions.yaml' 2026-05-11T15:53:42,834 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_abstract_algebra.yaml' 2026-05-11T15:53:42,835 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_anatomy.yaml' 2026-05-11T15:53:42,837 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_astronomy.yaml' 2026-05-11T15:53:42,838 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_business_ethics.yaml' 2026-05-11T15:53:42,839 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_clinical_knowledge.yaml' 2026-05-11T15:53:42,841 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_biology.yaml' 2026-05-11T15:53:42,842 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_chemistry.yaml' 2026-05-11T15:53:42,843 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_computer_science.yaml' 2026-05-11T15:53:42,844 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_mathematics.yaml' 2026-05-11T15:53:42,845 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_medicine.yaml' 2026-05-11T15:53:42,846 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_college_physics.yaml' 2026-05-11T15:53:42,847 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_computer_security.yaml' 2026-05-11T15:53:42,849 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_conceptual_physics.yaml' 2026-05-11T15:53:42,850 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_econometrics.yaml' 2026-05-11T15:53:42,851 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_electrical_engineering.yaml' 2026-05-11T15:53:42,852 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_elementary_mathematics.yaml' 2026-05-11T15:53:42,853 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_formal_logic.yaml' 2026-05-11T15:53:42,855 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_global_facts.yaml' 2026-05-11T15:53:42,856 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_biology.yaml' 2026-05-11T15:53:42,857 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_chemistry.yaml' 2026-05-11T15:53:42,858 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_computer_science.yaml' 2026-05-11T15:53:42,859 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_european_history.yaml' 2026-05-11T15:53:42,861 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_geography.yaml' 2026-05-11T15:53:42,862 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,863 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_macroeconomics.yaml' 2026-05-11T15:53:42,864 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_mathematics.yaml' 2026-05-11T15:53:42,865 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_microeconomics.yaml' 2026-05-11T15:53:42,867 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_physics.yaml' 2026-05-11T15:53:42,868 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_psychology.yaml' 2026-05-11T15:53:42,869 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_statistics.yaml' 2026-05-11T15:53:42,870 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_us_history.yaml' 2026-05-11T15:53:42,872 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_high_school_world_history.yaml' 2026-05-11T15:53:42,873 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_aging.yaml' 2026-05-11T15:53:42,874 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_human_sexuality.yaml' 2026-05-11T15:53:42,875 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_international_law.yaml' 2026-05-11T15:53:42,876 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_jurisprudence.yaml' 2026-05-11T15:53:42,878 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_logical_fallacies.yaml' 2026-05-11T15:53:42,879 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_machine_learning.yaml' 2026-05-11T15:53:42,880 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_management.yaml' 2026-05-11T15:53:42,882 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_marketing.yaml' 2026-05-11T15:53:42,883 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_medical_genetics.yaml' 2026-05-11T15:53:42,884 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_miscellaneous.yaml' 2026-05-11T15:53:42,885 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_disputes.yaml' 2026-05-11T15:53:42,887 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_moral_scenarios.yaml' 2026-05-11T15:53:42,888 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_nutrition.yaml' 2026-05-11T15:53:42,889 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_philosophy.yaml' 2026-05-11T15:53:42,890 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_prehistory.yaml' 2026-05-11T15:53:42,891 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_accounting.yaml' 2026-05-11T15:53:42,893 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_law.yaml' 2026-05-11T15:53:42,894 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_medicine.yaml' 2026-05-11T15:53:42,895 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_professional_psychology.yaml' 2026-05-11T15:53:42,896 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_public_relations.yaml' 2026-05-11T15:53:42,897 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_security_studies.yaml' 2026-05-11T15:53:42,898 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_sociology.yaml' 2026-05-11T15:53:42,900 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_us_foreign_policy.yaml' 2026-05-11T15:53:42,901 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_virology.yaml' 2026-05-11T15:53:42,902 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_fr_fr_world_religions.yaml' 2026-05-11T15:53:42,903 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_abstract_algebra.yaml' 2026-05-11T15:53:42,904 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_anatomy.yaml' 2026-05-11T15:53:42,906 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_astronomy.yaml' 2026-05-11T15:53:42,907 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_business_ethics.yaml' 2026-05-11T15:53:42,908 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_clinical_knowledge.yaml' 2026-05-11T15:53:42,909 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_biology.yaml' 2026-05-11T15:53:42,910 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_chemistry.yaml' 2026-05-11T15:53:42,912 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_computer_science.yaml' 2026-05-11T15:53:42,913 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_mathematics.yaml' 2026-05-11T15:53:42,914 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_medicine.yaml' 2026-05-11T15:53:42,915 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_college_physics.yaml' 2026-05-11T15:53:42,916 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_computer_security.yaml' 2026-05-11T15:53:42,917 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_conceptual_physics.yaml' 2026-05-11T15:53:42,919 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_econometrics.yaml' 2026-05-11T15:53:42,920 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_electrical_engineering.yaml' 2026-05-11T15:53:42,921 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_elementary_mathematics.yaml' 2026-05-11T15:53:42,922 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_formal_logic.yaml' 2026-05-11T15:53:42,924 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_global_facts.yaml' 2026-05-11T15:53:42,925 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_biology.yaml' 2026-05-11T15:53:42,926 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_chemistry.yaml' 2026-05-11T15:53:42,927 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_computer_science.yaml' 2026-05-11T15:53:42,929 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_european_history.yaml' 2026-05-11T15:53:42,930 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_geography.yaml' 2026-05-11T15:53:42,931 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,932 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_macroeconomics.yaml' 2026-05-11T15:53:42,933 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_mathematics.yaml' 2026-05-11T15:53:42,935 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_microeconomics.yaml' 2026-05-11T15:53:42,936 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_physics.yaml' 2026-05-11T15:53:42,937 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_psychology.yaml' 2026-05-11T15:53:42,938 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_statistics.yaml' 2026-05-11T15:53:42,939 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_us_history.yaml' 2026-05-11T15:53:42,940 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_high_school_world_history.yaml' 2026-05-11T15:53:42,942 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_aging.yaml' 2026-05-11T15:53:42,943 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_human_sexuality.yaml' 2026-05-11T15:53:42,944 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_international_law.yaml' 2026-05-11T15:53:42,945 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_jurisprudence.yaml' 2026-05-11T15:53:42,946 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_logical_fallacies.yaml' 2026-05-11T15:53:42,947 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_machine_learning.yaml' 2026-05-11T15:53:42,948 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_management.yaml' 2026-05-11T15:53:42,949 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_marketing.yaml' 2026-05-11T15:53:42,951 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_medical_genetics.yaml' 2026-05-11T15:53:42,952 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_miscellaneous.yaml' 2026-05-11T15:53:42,953 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_disputes.yaml' 2026-05-11T15:53:42,954 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_moral_scenarios.yaml' 2026-05-11T15:53:42,955 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_nutrition.yaml' 2026-05-11T15:53:42,956 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_philosophy.yaml' 2026-05-11T15:53:42,958 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_prehistory.yaml' 2026-05-11T15:53:42,959 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_accounting.yaml' 2026-05-11T15:53:42,960 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_law.yaml' 2026-05-11T15:53:42,961 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_medicine.yaml' 2026-05-11T15:53:42,962 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_professional_psychology.yaml' 2026-05-11T15:53:42,964 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_public_relations.yaml' 2026-05-11T15:53:42,965 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_security_studies.yaml' 2026-05-11T15:53:42,966 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_sociology.yaml' 2026-05-11T15:53:42,967 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_us_foreign_policy.yaml' 2026-05-11T15:53:42,968 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_virology.yaml' 2026-05-11T15:53:42,970 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_hi_in_world_religions.yaml' 2026-05-11T15:53:42,971 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_abstract_algebra.yaml' 2026-05-11T15:53:42,972 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_anatomy.yaml' 2026-05-11T15:53:42,973 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_astronomy.yaml' 2026-05-11T15:53:42,974 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_business_ethics.yaml' 2026-05-11T15:53:42,975 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_clinical_knowledge.yaml' 2026-05-11T15:53:42,977 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_biology.yaml' 2026-05-11T15:53:42,978 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_chemistry.yaml' 2026-05-11T15:53:42,979 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_computer_science.yaml' 2026-05-11T15:53:42,981 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_mathematics.yaml' 2026-05-11T15:53:42,982 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_medicine.yaml' 2026-05-11T15:53:42,983 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_college_physics.yaml' 2026-05-11T15:53:42,984 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_computer_security.yaml' 2026-05-11T15:53:42,985 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_conceptual_physics.yaml' 2026-05-11T15:53:42,986 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_econometrics.yaml' 2026-05-11T15:53:42,987 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_electrical_engineering.yaml' 2026-05-11T15:53:42,988 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_elementary_mathematics.yaml' 2026-05-11T15:53:42,990 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_formal_logic.yaml' 2026-05-11T15:53:42,991 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_global_facts.yaml' 2026-05-11T15:53:42,992 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_biology.yaml' 2026-05-11T15:53:42,993 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_chemistry.yaml' 2026-05-11T15:53:42,994 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_computer_science.yaml' 2026-05-11T15:53:42,995 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_european_history.yaml' 2026-05-11T15:53:42,996 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_geography.yaml' 2026-05-11T15:53:42,997 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_government_and_politics.yaml' 2026-05-11T15:53:42,999 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,000 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_mathematics.yaml' 2026-05-11T15:53:43,001 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_microeconomics.yaml' 2026-05-11T15:53:43,002 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_physics.yaml' 2026-05-11T15:53:43,003 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_psychology.yaml' 2026-05-11T15:53:43,005 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_statistics.yaml' 2026-05-11T15:53:43,006 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_us_history.yaml' 2026-05-11T15:53:43,007 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_high_school_world_history.yaml' 2026-05-11T15:53:43,008 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_aging.yaml' 2026-05-11T15:53:43,009 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_human_sexuality.yaml' 2026-05-11T15:53:43,011 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_international_law.yaml' 2026-05-11T15:53:43,012 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_jurisprudence.yaml' 2026-05-11T15:53:43,013 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_logical_fallacies.yaml' 2026-05-11T15:53:43,014 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_machine_learning.yaml' 2026-05-11T15:53:43,015 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_management.yaml' 2026-05-11T15:53:43,016 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_marketing.yaml' 2026-05-11T15:53:43,018 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_medical_genetics.yaml' 2026-05-11T15:53:43,019 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_miscellaneous.yaml' 2026-05-11T15:53:43,020 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_disputes.yaml' 2026-05-11T15:53:43,021 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_moral_scenarios.yaml' 2026-05-11T15:53:43,023 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_nutrition.yaml' 2026-05-11T15:53:43,024 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_philosophy.yaml' 2026-05-11T15:53:43,025 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_prehistory.yaml' 2026-05-11T15:53:43,026 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_accounting.yaml' 2026-05-11T15:53:43,027 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_law.yaml' 2026-05-11T15:53:43,028 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_medicine.yaml' 2026-05-11T15:53:43,029 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_professional_psychology.yaml' 2026-05-11T15:53:43,030 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_public_relations.yaml' 2026-05-11T15:53:43,031 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_security_studies.yaml' 2026-05-11T15:53:43,032 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_sociology.yaml' 2026-05-11T15:53:43,034 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_us_foreign_policy.yaml' 2026-05-11T15:53:43,035 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_virology.yaml' 2026-05-11T15:53:43,036 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_id_id_world_religions.yaml' 2026-05-11T15:53:43,037 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_abstract_algebra.yaml' 2026-05-11T15:53:43,038 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_anatomy.yaml' 2026-05-11T15:53:43,039 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_astronomy.yaml' 2026-05-11T15:53:43,040 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_business_ethics.yaml' 2026-05-11T15:53:43,042 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_clinical_knowledge.yaml' 2026-05-11T15:53:43,043 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_biology.yaml' 2026-05-11T15:53:43,044 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_chemistry.yaml' 2026-05-11T15:53:43,045 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_computer_science.yaml' 2026-05-11T15:53:43,046 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_mathematics.yaml' 2026-05-11T15:53:43,047 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_medicine.yaml' 2026-05-11T15:53:43,049 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_college_physics.yaml' 2026-05-11T15:53:43,050 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_computer_security.yaml' 2026-05-11T15:53:43,051 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_conceptual_physics.yaml' 2026-05-11T15:53:43,052 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_econometrics.yaml' 2026-05-11T15:53:43,053 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_electrical_engineering.yaml' 2026-05-11T15:53:43,054 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_elementary_mathematics.yaml' 2026-05-11T15:53:43,056 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_formal_logic.yaml' 2026-05-11T15:53:43,057 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_global_facts.yaml' 2026-05-11T15:53:43,058 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_biology.yaml' 2026-05-11T15:53:43,059 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_chemistry.yaml' 2026-05-11T15:53:43,061 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_computer_science.yaml' 2026-05-11T15:53:43,062 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_european_history.yaml' 2026-05-11T15:53:43,063 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_geography.yaml' 2026-05-11T15:53:43,064 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,065 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,066 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_mathematics.yaml' 2026-05-11T15:53:43,068 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_microeconomics.yaml' 2026-05-11T15:53:43,069 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_physics.yaml' 2026-05-11T15:53:43,070 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_psychology.yaml' 2026-05-11T15:53:43,071 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_statistics.yaml' 2026-05-11T15:53:43,072 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_us_history.yaml' 2026-05-11T15:53:43,074 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_high_school_world_history.yaml' 2026-05-11T15:53:43,075 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_aging.yaml' 2026-05-11T15:53:43,076 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_human_sexuality.yaml' 2026-05-11T15:53:43,077 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_international_law.yaml' 2026-05-11T15:53:43,078 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_jurisprudence.yaml' 2026-05-11T15:53:43,079 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_logical_fallacies.yaml' 2026-05-11T15:53:43,080 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_machine_learning.yaml' 2026-05-11T15:53:43,082 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_management.yaml' 2026-05-11T15:53:43,083 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_marketing.yaml' 2026-05-11T15:53:43,084 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_medical_genetics.yaml' 2026-05-11T15:53:43,085 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_miscellaneous.yaml' 2026-05-11T15:53:43,086 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_disputes.yaml' 2026-05-11T15:53:43,087 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_moral_scenarios.yaml' 2026-05-11T15:53:43,089 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_nutrition.yaml' 2026-05-11T15:53:43,090 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_philosophy.yaml' 2026-05-11T15:53:43,091 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_prehistory.yaml' 2026-05-11T15:53:43,092 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_accounting.yaml' 2026-05-11T15:53:43,093 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_law.yaml' 2026-05-11T15:53:43,094 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_medicine.yaml' 2026-05-11T15:53:43,096 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_professional_psychology.yaml' 2026-05-11T15:53:43,097 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_public_relations.yaml' 2026-05-11T15:53:43,098 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_security_studies.yaml' 2026-05-11T15:53:43,099 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_sociology.yaml' 2026-05-11T15:53:43,100 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_us_foreign_policy.yaml' 2026-05-11T15:53:43,101 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_virology.yaml' 2026-05-11T15:53:43,103 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_it_it_world_religions.yaml' 2026-05-11T15:53:43,104 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_abstract_algebra.yaml' 2026-05-11T15:53:43,105 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_anatomy.yaml' 2026-05-11T15:53:43,106 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_astronomy.yaml' 2026-05-11T15:53:43,107 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_business_ethics.yaml' 2026-05-11T15:53:43,109 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_clinical_knowledge.yaml' 2026-05-11T15:53:43,110 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_biology.yaml' 2026-05-11T15:53:43,111 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_chemistry.yaml' 2026-05-11T15:53:43,112 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_computer_science.yaml' 2026-05-11T15:53:43,114 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_mathematics.yaml' 2026-05-11T15:53:43,115 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_medicine.yaml' 2026-05-11T15:53:43,116 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_college_physics.yaml' 2026-05-11T15:53:43,117 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_computer_security.yaml' 2026-05-11T15:53:43,118 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_conceptual_physics.yaml' 2026-05-11T15:53:43,120 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_econometrics.yaml' 2026-05-11T15:53:43,121 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_electrical_engineering.yaml' 2026-05-11T15:53:43,122 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_elementary_mathematics.yaml' 2026-05-11T15:53:43,123 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_formal_logic.yaml' 2026-05-11T15:53:43,124 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_global_facts.yaml' 2026-05-11T15:53:43,125 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_biology.yaml' 2026-05-11T15:53:43,126 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_chemistry.yaml' 2026-05-11T15:53:43,127 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_computer_science.yaml' 2026-05-11T15:53:43,129 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_european_history.yaml' 2026-05-11T15:53:43,130 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_geography.yaml' 2026-05-11T15:53:43,131 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,132 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,133 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_mathematics.yaml' 2026-05-11T15:53:43,135 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_microeconomics.yaml' 2026-05-11T15:53:43,136 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_physics.yaml' 2026-05-11T15:53:43,137 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_psychology.yaml' 2026-05-11T15:53:43,138 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_statistics.yaml' 2026-05-11T15:53:43,139 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_us_history.yaml' 2026-05-11T15:53:43,141 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_high_school_world_history.yaml' 2026-05-11T15:53:43,142 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_aging.yaml' 2026-05-11T15:53:43,143 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_human_sexuality.yaml' 2026-05-11T15:53:43,144 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_international_law.yaml' 2026-05-11T15:53:43,145 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_jurisprudence.yaml' 2026-05-11T15:53:43,146 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_logical_fallacies.yaml' 2026-05-11T15:53:43,147 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_machine_learning.yaml' 2026-05-11T15:53:43,149 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_management.yaml' 2026-05-11T15:53:43,150 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_marketing.yaml' 2026-05-11T15:53:43,151 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_medical_genetics.yaml' 2026-05-11T15:53:43,152 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_miscellaneous.yaml' 2026-05-11T15:53:43,153 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_disputes.yaml' 2026-05-11T15:53:43,154 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_moral_scenarios.yaml' 2026-05-11T15:53:43,156 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_nutrition.yaml' 2026-05-11T15:53:43,157 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_philosophy.yaml' 2026-05-11T15:53:43,158 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_prehistory.yaml' 2026-05-11T15:53:43,159 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_accounting.yaml' 2026-05-11T15:53:43,160 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_law.yaml' 2026-05-11T15:53:43,161 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_medicine.yaml' 2026-05-11T15:53:43,163 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_professional_psychology.yaml' 2026-05-11T15:53:43,164 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_public_relations.yaml' 2026-05-11T15:53:43,165 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_security_studies.yaml' 2026-05-11T15:53:43,166 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_sociology.yaml' 2026-05-11T15:53:43,167 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_us_foreign_policy.yaml' 2026-05-11T15:53:43,168 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_virology.yaml' 2026-05-11T15:53:43,169 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ja_jp_world_religions.yaml' 2026-05-11T15:53:43,170 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_abstract_algebra.yaml' 2026-05-11T15:53:43,172 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_anatomy.yaml' 2026-05-11T15:53:43,173 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_astronomy.yaml' 2026-05-11T15:53:43,174 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_business_ethics.yaml' 2026-05-11T15:53:43,175 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_clinical_knowledge.yaml' 2026-05-11T15:53:43,176 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_biology.yaml' 2026-05-11T15:53:43,178 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_chemistry.yaml' 2026-05-11T15:53:43,179 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_computer_science.yaml' 2026-05-11T15:53:43,180 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_mathematics.yaml' 2026-05-11T15:53:43,181 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_medicine.yaml' 2026-05-11T15:53:43,182 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_college_physics.yaml' 2026-05-11T15:53:43,184 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_computer_security.yaml' 2026-05-11T15:53:43,185 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_conceptual_physics.yaml' 2026-05-11T15:53:43,186 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_econometrics.yaml' 2026-05-11T15:53:43,187 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_electrical_engineering.yaml' 2026-05-11T15:53:43,189 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_elementary_mathematics.yaml' 2026-05-11T15:53:43,190 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_formal_logic.yaml' 2026-05-11T15:53:43,191 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_global_facts.yaml' 2026-05-11T15:53:43,192 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_biology.yaml' 2026-05-11T15:53:43,194 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_chemistry.yaml' 2026-05-11T15:53:43,195 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_computer_science.yaml' 2026-05-11T15:53:43,196 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_european_history.yaml' 2026-05-11T15:53:43,197 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_geography.yaml' 2026-05-11T15:53:43,198 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,200 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,201 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_mathematics.yaml' 2026-05-11T15:53:43,202 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_microeconomics.yaml' 2026-05-11T15:53:43,204 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_physics.yaml' 2026-05-11T15:53:43,205 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_psychology.yaml' 2026-05-11T15:53:43,207 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_statistics.yaml' 2026-05-11T15:53:43,208 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_us_history.yaml' 2026-05-11T15:53:43,209 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_high_school_world_history.yaml' 2026-05-11T15:53:43,210 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_aging.yaml' 2026-05-11T15:53:43,211 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_human_sexuality.yaml' 2026-05-11T15:53:43,212 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_international_law.yaml' 2026-05-11T15:53:43,214 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_jurisprudence.yaml' 2026-05-11T15:53:43,215 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_logical_fallacies.yaml' 2026-05-11T15:53:43,216 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_machine_learning.yaml' 2026-05-11T15:53:43,217 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_management.yaml' 2026-05-11T15:53:43,218 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_marketing.yaml' 2026-05-11T15:53:43,220 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_medical_genetics.yaml' 2026-05-11T15:53:43,221 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_miscellaneous.yaml' 2026-05-11T15:53:43,222 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_disputes.yaml' 2026-05-11T15:53:43,223 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_moral_scenarios.yaml' 2026-05-11T15:53:43,224 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_nutrition.yaml' 2026-05-11T15:53:43,226 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_philosophy.yaml' 2026-05-11T15:53:43,227 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_prehistory.yaml' 2026-05-11T15:53:43,228 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_accounting.yaml' 2026-05-11T15:53:43,229 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_law.yaml' 2026-05-11T15:53:43,230 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_medicine.yaml' 2026-05-11T15:53:43,231 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_professional_psychology.yaml' 2026-05-11T15:53:43,233 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_public_relations.yaml' 2026-05-11T15:53:43,234 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_security_studies.yaml' 2026-05-11T15:53:43,235 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_sociology.yaml' 2026-05-11T15:53:43,236 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_us_foreign_policy.yaml' 2026-05-11T15:53:43,237 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_virology.yaml' 2026-05-11T15:53:43,238 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_ko_kr_world_religions.yaml' 2026-05-11T15:53:43,240 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_abstract_algebra.yaml' 2026-05-11T15:53:43,241 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_anatomy.yaml' 2026-05-11T15:53:43,242 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_astronomy.yaml' 2026-05-11T15:53:43,243 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_business_ethics.yaml' 2026-05-11T15:53:43,245 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_clinical_knowledge.yaml' 2026-05-11T15:53:43,246 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_biology.yaml' 2026-05-11T15:53:43,247 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_chemistry.yaml' 2026-05-11T15:53:43,248 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_computer_science.yaml' 2026-05-11T15:53:43,249 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_mathematics.yaml' 2026-05-11T15:53:43,250 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_medicine.yaml' 2026-05-11T15:53:43,252 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_college_physics.yaml' 2026-05-11T15:53:43,253 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_computer_security.yaml' 2026-05-11T15:53:43,254 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_conceptual_physics.yaml' 2026-05-11T15:53:43,255 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_econometrics.yaml' 2026-05-11T15:53:43,256 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_electrical_engineering.yaml' 2026-05-11T15:53:43,258 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_elementary_mathematics.yaml' 2026-05-11T15:53:43,259 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_formal_logic.yaml' 2026-05-11T15:53:43,260 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_global_facts.yaml' 2026-05-11T15:53:43,261 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_biology.yaml' 2026-05-11T15:53:43,262 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_chemistry.yaml' 2026-05-11T15:53:43,263 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_computer_science.yaml' 2026-05-11T15:53:43,264 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_european_history.yaml' 2026-05-11T15:53:43,265 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_geography.yaml' 2026-05-11T15:53:43,267 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,268 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,269 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_mathematics.yaml' 2026-05-11T15:53:43,270 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_microeconomics.yaml' 2026-05-11T15:53:43,271 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_physics.yaml' 2026-05-11T15:53:43,272 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_psychology.yaml' 2026-05-11T15:53:43,273 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_statistics.yaml' 2026-05-11T15:53:43,275 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_us_history.yaml' 2026-05-11T15:53:43,276 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_high_school_world_history.yaml' 2026-05-11T15:53:43,277 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_aging.yaml' 2026-05-11T15:53:43,278 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_human_sexuality.yaml' 2026-05-11T15:53:43,279 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_international_law.yaml' 2026-05-11T15:53:43,281 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_jurisprudence.yaml' 2026-05-11T15:53:43,282 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_logical_fallacies.yaml' 2026-05-11T15:53:43,283 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_machine_learning.yaml' 2026-05-11T15:53:43,284 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_management.yaml' 2026-05-11T15:53:43,285 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_marketing.yaml' 2026-05-11T15:53:43,287 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_medical_genetics.yaml' 2026-05-11T15:53:43,288 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_miscellaneous.yaml' 2026-05-11T15:53:43,289 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_disputes.yaml' 2026-05-11T15:53:43,290 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_moral_scenarios.yaml' 2026-05-11T15:53:43,291 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_nutrition.yaml' 2026-05-11T15:53:43,293 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_philosophy.yaml' 2026-05-11T15:53:43,294 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_prehistory.yaml' 2026-05-11T15:53:43,295 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_accounting.yaml' 2026-05-11T15:53:43,296 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_law.yaml' 2026-05-11T15:53:43,297 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_medicine.yaml' 2026-05-11T15:53:43,299 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_professional_psychology.yaml' 2026-05-11T15:53:43,300 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_public_relations.yaml' 2026-05-11T15:53:43,301 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_security_studies.yaml' 2026-05-11T15:53:43,303 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_sociology.yaml' 2026-05-11T15:53:43,304 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_us_foreign_policy.yaml' 2026-05-11T15:53:43,305 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_virology.yaml' 2026-05-11T15:53:43,306 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_pt_br_world_religions.yaml' 2026-05-11T15:53:43,307 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_abstract_algebra.yaml' 2026-05-11T15:53:43,308 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_anatomy.yaml' 2026-05-11T15:53:43,309 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_astronomy.yaml' 2026-05-11T15:53:43,310 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_business_ethics.yaml' 2026-05-11T15:53:43,311 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_clinical_knowledge.yaml' 2026-05-11T15:53:43,312 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_biology.yaml' 2026-05-11T15:53:43,313 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_chemistry.yaml' 2026-05-11T15:53:43,315 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_computer_science.yaml' 2026-05-11T15:53:43,316 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_mathematics.yaml' 2026-05-11T15:53:43,317 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_medicine.yaml' 2026-05-11T15:53:43,318 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_college_physics.yaml' 2026-05-11T15:53:43,319 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_computer_security.yaml' 2026-05-11T15:53:43,321 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_conceptual_physics.yaml' 2026-05-11T15:53:43,322 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_econometrics.yaml' 2026-05-11T15:53:43,323 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_electrical_engineering.yaml' 2026-05-11T15:53:43,324 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_elementary_mathematics.yaml' 2026-05-11T15:53:43,325 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_formal_logic.yaml' 2026-05-11T15:53:43,326 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_global_facts.yaml' 2026-05-11T15:53:43,327 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_biology.yaml' 2026-05-11T15:53:43,329 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_chemistry.yaml' 2026-05-11T15:53:43,330 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_computer_science.yaml' 2026-05-11T15:53:43,331 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_european_history.yaml' 2026-05-11T15:53:43,332 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_geography.yaml' 2026-05-11T15:53:43,333 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,335 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,336 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_mathematics.yaml' 2026-05-11T15:53:43,337 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_microeconomics.yaml' 2026-05-11T15:53:43,338 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_physics.yaml' 2026-05-11T15:53:43,339 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_psychology.yaml' 2026-05-11T15:53:43,341 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_statistics.yaml' 2026-05-11T15:53:43,342 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_us_history.yaml' 2026-05-11T15:53:43,343 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_high_school_world_history.yaml' 2026-05-11T15:53:43,344 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_aging.yaml' 2026-05-11T15:53:43,345 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_human_sexuality.yaml' 2026-05-11T15:53:43,347 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_international_law.yaml' 2026-05-11T15:53:43,348 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_jurisprudence.yaml' 2026-05-11T15:53:43,349 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_logical_fallacies.yaml' 2026-05-11T15:53:43,350 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_machine_learning.yaml' 2026-05-11T15:53:43,351 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_management.yaml' 2026-05-11T15:53:43,352 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_marketing.yaml' 2026-05-11T15:53:43,353 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_medical_genetics.yaml' 2026-05-11T15:53:43,355 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_miscellaneous.yaml' 2026-05-11T15:53:43,356 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_disputes.yaml' 2026-05-11T15:53:43,357 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_moral_scenarios.yaml' 2026-05-11T15:53:43,358 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_nutrition.yaml' 2026-05-11T15:53:43,359 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_philosophy.yaml' 2026-05-11T15:53:43,360 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_prehistory.yaml' 2026-05-11T15:53:43,361 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_accounting.yaml' 2026-05-11T15:53:43,363 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_law.yaml' 2026-05-11T15:53:43,364 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_medicine.yaml' 2026-05-11T15:53:43,365 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_professional_psychology.yaml' 2026-05-11T15:53:43,366 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_public_relations.yaml' 2026-05-11T15:53:43,367 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_security_studies.yaml' 2026-05-11T15:53:43,368 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_sociology.yaml' 2026-05-11T15:53:43,370 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_us_foreign_policy.yaml' 2026-05-11T15:53:43,371 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_virology.yaml' 2026-05-11T15:53:43,372 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_sw_ke_world_religions.yaml' 2026-05-11T15:53:43,373 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_abstract_algebra.yaml' 2026-05-11T15:53:43,375 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_anatomy.yaml' 2026-05-11T15:53:43,376 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_astronomy.yaml' 2026-05-11T15:53:43,377 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_business_ethics.yaml' 2026-05-11T15:53:43,378 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_clinical_knowledge.yaml' 2026-05-11T15:53:43,379 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_biology.yaml' 2026-05-11T15:53:43,381 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_chemistry.yaml' 2026-05-11T15:53:43,382 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_computer_science.yaml' 2026-05-11T15:53:43,383 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_mathematics.yaml' 2026-05-11T15:53:43,384 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_medicine.yaml' 2026-05-11T15:53:43,385 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_college_physics.yaml' 2026-05-11T15:53:43,386 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_computer_security.yaml' 2026-05-11T15:53:43,388 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_conceptual_physics.yaml' 2026-05-11T15:53:43,389 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_econometrics.yaml' 2026-05-11T15:53:43,390 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_electrical_engineering.yaml' 2026-05-11T15:53:43,391 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_elementary_mathematics.yaml' 2026-05-11T15:53:43,392 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_formal_logic.yaml' 2026-05-11T15:53:43,393 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_global_facts.yaml' 2026-05-11T15:53:43,395 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_biology.yaml' 2026-05-11T15:53:43,396 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_chemistry.yaml' 2026-05-11T15:53:43,397 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_computer_science.yaml' 2026-05-11T15:53:43,398 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_european_history.yaml' 2026-05-11T15:53:43,399 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_geography.yaml' 2026-05-11T15:53:43,400 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,401 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,402 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_mathematics.yaml' 2026-05-11T15:53:43,403 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_microeconomics.yaml' 2026-05-11T15:53:43,404 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_physics.yaml' 2026-05-11T15:53:43,406 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_psychology.yaml' 2026-05-11T15:53:43,407 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_statistics.yaml' 2026-05-11T15:53:43,408 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_us_history.yaml' 2026-05-11T15:53:43,409 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_high_school_world_history.yaml' 2026-05-11T15:53:43,411 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_aging.yaml' 2026-05-11T15:53:43,412 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_human_sexuality.yaml' 2026-05-11T15:53:43,413 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_international_law.yaml' 2026-05-11T15:53:43,414 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_jurisprudence.yaml' 2026-05-11T15:53:43,415 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_logical_fallacies.yaml' 2026-05-11T15:53:43,416 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_machine_learning.yaml' 2026-05-11T15:53:43,418 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_management.yaml' 2026-05-11T15:53:43,419 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_marketing.yaml' 2026-05-11T15:53:43,420 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_medical_genetics.yaml' 2026-05-11T15:53:43,421 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_miscellaneous.yaml' 2026-05-11T15:53:43,423 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_disputes.yaml' 2026-05-11T15:53:43,424 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_moral_scenarios.yaml' 2026-05-11T15:53:43,425 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_nutrition.yaml' 2026-05-11T15:53:43,426 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_philosophy.yaml' 2026-05-11T15:53:43,427 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_prehistory.yaml' 2026-05-11T15:53:43,429 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_accounting.yaml' 2026-05-11T15:53:43,430 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_law.yaml' 2026-05-11T15:53:43,431 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_medicine.yaml' 2026-05-11T15:53:43,432 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_professional_psychology.yaml' 2026-05-11T15:53:43,434 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_public_relations.yaml' 2026-05-11T15:53:43,435 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_security_studies.yaml' 2026-05-11T15:53:43,436 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_sociology.yaml' 2026-05-11T15:53:43,437 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_us_foreign_policy.yaml' 2026-05-11T15:53:43,438 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_virology.yaml' 2026-05-11T15:53:43,439 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_yo_ng_world_religions.yaml' 2026-05-11T15:53:43,440 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_abstract_algebra.yaml' 2026-05-11T15:53:43,442 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_anatomy.yaml' 2026-05-11T15:53:43,443 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_astronomy.yaml' 2026-05-11T15:53:43,444 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_business_ethics.yaml' 2026-05-11T15:53:43,445 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_clinical_knowledge.yaml' 2026-05-11T15:53:43,446 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_biology.yaml' 2026-05-11T15:53:43,447 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_chemistry.yaml' 2026-05-11T15:53:43,448 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_computer_science.yaml' 2026-05-11T15:53:43,450 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_mathematics.yaml' 2026-05-11T15:53:43,451 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_medicine.yaml' 2026-05-11T15:53:43,452 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_college_physics.yaml' 2026-05-11T15:53:43,453 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_computer_security.yaml' 2026-05-11T15:53:43,454 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_conceptual_physics.yaml' 2026-05-11T15:53:43,455 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_econometrics.yaml' 2026-05-11T15:53:43,456 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_electrical_engineering.yaml' 2026-05-11T15:53:43,457 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_elementary_mathematics.yaml' 2026-05-11T15:53:43,459 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_formal_logic.yaml' 2026-05-11T15:53:43,460 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_global_facts.yaml' 2026-05-11T15:53:43,461 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_biology.yaml' 2026-05-11T15:53:43,462 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_chemistry.yaml' 2026-05-11T15:53:43,464 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_computer_science.yaml' 2026-05-11T15:53:43,465 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_european_history.yaml' 2026-05-11T15:53:43,466 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_geography.yaml' 2026-05-11T15:53:43,467 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_government_and_politics.yaml' 2026-05-11T15:53:43,468 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_macroeconomics.yaml' 2026-05-11T15:53:43,470 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_mathematics.yaml' 2026-05-11T15:53:43,471 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_microeconomics.yaml' 2026-05-11T15:53:43,472 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_physics.yaml' 2026-05-11T15:53:43,473 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_psychology.yaml' 2026-05-11T15:53:43,474 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_statistics.yaml' 2026-05-11T15:53:43,476 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_us_history.yaml' 2026-05-11T15:53:43,477 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_high_school_world_history.yaml' 2026-05-11T15:53:43,478 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_aging.yaml' 2026-05-11T15:53:43,479 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_human_sexuality.yaml' 2026-05-11T15:53:43,480 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_international_law.yaml' 2026-05-11T15:53:43,482 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_jurisprudence.yaml' 2026-05-11T15:53:43,483 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_logical_fallacies.yaml' 2026-05-11T15:53:43,484 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_machine_learning.yaml' 2026-05-11T15:53:43,485 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_management.yaml' 2026-05-11T15:53:43,486 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_marketing.yaml' 2026-05-11T15:53:43,487 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_medical_genetics.yaml' 2026-05-11T15:53:43,489 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_miscellaneous.yaml' 2026-05-11T15:53:43,490 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_disputes.yaml' 2026-05-11T15:53:43,491 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_moral_scenarios.yaml' 2026-05-11T15:53:43,492 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_nutrition.yaml' 2026-05-11T15:53:43,493 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_philosophy.yaml' 2026-05-11T15:53:43,494 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_prehistory.yaml' 2026-05-11T15:53:43,495 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_accounting.yaml' 2026-05-11T15:53:43,496 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_law.yaml' 2026-05-11T15:53:43,497 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_medicine.yaml' 2026-05-11T15:53:43,499 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_professional_psychology.yaml' 2026-05-11T15:53:43,500 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_public_relations.yaml' 2026-05-11T15:53:43,501 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_security_studies.yaml' 2026-05-11T15:53:43,502 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_sociology.yaml' 2026-05-11T15:53:43,503 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_us_foreign_policy.yaml' 2026-05-11T15:53:43,505 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_virology.yaml' 2026-05-11T15:53:43,506 adding 'lm_eval/tasks/openai-mmmlu/default/mmmlu_zh_cn_world_religions.yaml' 2026-05-11T15:53:43,507 adding 'lm_eval/tasks/openai-mmmlu/default/utils.py' 2026-05-11T15:53:43,509 adding 'lm_eval/tasks/openbookqa/README.md' 2026-05-11T15:53:43,510 adding 'lm_eval/tasks/openbookqa/openbookqa.yaml' 2026-05-11T15:53:43,512 adding 'lm_eval/tasks/paloma/README.md' 2026-05-11T15:53:43,513 adding 'lm_eval/tasks/paloma/_paloma_template' 2026-05-11T15:53:43,515 adding 'lm_eval/tasks/paloma/paloma_4chan_meta_sep.yaml' 2026-05-11T15:53:43,516 adding 'lm_eval/tasks/paloma/paloma_c4_100_domains.yaml' 2026-05-11T15:53:43,517 adding 'lm_eval/tasks/paloma/paloma_c4_en.yaml' 2026-05-11T15:53:43,518 adding 'lm_eval/tasks/paloma/paloma_dolma-v1_5.yaml' 2026-05-11T15:53:43,520 adding 'lm_eval/tasks/paloma/paloma_dolma_100_programing_languages.yaml' 2026-05-11T15:53:43,521 adding 'lm_eval/tasks/paloma/paloma_dolma_100_subreddits.yaml' 2026-05-11T15:53:43,522 adding 'lm_eval/tasks/paloma/paloma_falcon-refinedweb.yaml' 2026-05-11T15:53:43,523 adding 'lm_eval/tasks/paloma/paloma_gab.yaml' 2026-05-11T15:53:43,524 adding 'lm_eval/tasks/paloma/paloma_m2d2_s2orc_unsplit.yaml' 2026-05-11T15:53:43,526 adding 'lm_eval/tasks/paloma/paloma_m2d2_wikipedia_unsplit.yaml' 2026-05-11T15:53:43,527 adding 'lm_eval/tasks/paloma/paloma_manosphere_meta_sep.yaml' 2026-05-11T15:53:43,528 adding 'lm_eval/tasks/paloma/paloma_mc4.yaml' 2026-05-11T15:53:43,529 adding 'lm_eval/tasks/paloma/paloma_ptb.yaml' 2026-05-11T15:53:43,530 adding 'lm_eval/tasks/paloma/paloma_redpajama.yaml' 2026-05-11T15:53:43,531 adding 'lm_eval/tasks/paloma/paloma_twitterAAE_HELM_fixed.yaml' 2026-05-11T15:53:43,533 adding 'lm_eval/tasks/paloma/paloma_utils.py' 2026-05-11T15:53:43,534 adding 'lm_eval/tasks/paloma/paloma_wikitext_103.yaml' 2026-05-11T15:53:43,536 adding 'lm_eval/tasks/paws-x/README.md' 2026-05-11T15:53:43,537 adding 'lm_eval/tasks/paws-x/_generate_config.py' 2026-05-11T15:53:43,538 adding 'lm_eval/tasks/paws-x/_pawsx.yaml' 2026-05-11T15:53:43,540 adding 'lm_eval/tasks/paws-x/paws_de.yaml' 2026-05-11T15:53:43,541 adding 'lm_eval/tasks/paws-x/paws_en.yaml' 2026-05-11T15:53:43,542 adding 'lm_eval/tasks/paws-x/paws_es.yaml' 2026-05-11T15:53:43,543 adding 'lm_eval/tasks/paws-x/paws_fr.yaml' 2026-05-11T15:53:43,544 adding 'lm_eval/tasks/paws-x/paws_ja.yaml' 2026-05-11T15:53:43,545 adding 'lm_eval/tasks/paws-x/paws_ko.yaml' 2026-05-11T15:53:43,546 adding 'lm_eval/tasks/paws-x/paws_zh.yaml' 2026-05-11T15:53:43,547 adding 'lm_eval/tasks/paws-x/pawsx_template_yaml' 2026-05-11T15:53:43,549 adding 'lm_eval/tasks/paws-x/utils.py' 2026-05-11T15:53:43,551 adding 'lm_eval/tasks/pile/README.md' 2026-05-11T15:53:43,552 adding 'lm_eval/tasks/pile/pile_arxiv.yaml' 2026-05-11T15:53:43,553 adding 'lm_eval/tasks/pile/pile_bookcorpus2.yaml' 2026-05-11T15:53:43,554 adding 'lm_eval/tasks/pile/pile_books3.yaml' 2026-05-11T15:53:43,555 adding 'lm_eval/tasks/pile/pile_dm-mathematics.yaml' 2026-05-11T15:53:43,556 adding 'lm_eval/tasks/pile/pile_enron.yaml' 2026-05-11T15:53:43,557 adding 'lm_eval/tasks/pile/pile_europarl.yaml' 2026-05-11T15:53:43,559 adding 'lm_eval/tasks/pile/pile_freelaw.yaml' 2026-05-11T15:53:43,560 adding 'lm_eval/tasks/pile/pile_github.yaml' 2026-05-11T15:53:43,561 adding 'lm_eval/tasks/pile/pile_gutenberg.yaml' 2026-05-11T15:53:43,562 adding 'lm_eval/tasks/pile/pile_hackernews.yaml' 2026-05-11T15:53:43,563 adding 'lm_eval/tasks/pile/pile_nih-exporter.yaml' 2026-05-11T15:53:43,564 adding 'lm_eval/tasks/pile/pile_opensubtitles.yaml' 2026-05-11T15:53:43,565 adding 'lm_eval/tasks/pile/pile_openwebtext2.yaml' 2026-05-11T15:53:43,566 adding 'lm_eval/tasks/pile/pile_philpapers.yaml' 2026-05-11T15:53:43,567 adding 'lm_eval/tasks/pile/pile_pile-cc.yaml' 2026-05-11T15:53:43,568 adding 'lm_eval/tasks/pile/pile_pubmed-abstracts.yaml' 2026-05-11T15:53:43,569 adding 'lm_eval/tasks/pile/pile_pubmed-central.yaml' 2026-05-11T15:53:43,570 adding 'lm_eval/tasks/pile/pile_stackexchange.yaml' 2026-05-11T15:53:43,572 adding 'lm_eval/tasks/pile/pile_ubuntu-irc.yaml' 2026-05-11T15:53:43,573 adding 'lm_eval/tasks/pile/pile_uspto.yaml' 2026-05-11T15:53:43,574 adding 'lm_eval/tasks/pile/pile_wikipedia.yaml' 2026-05-11T15:53:43,575 adding 'lm_eval/tasks/pile/pile_youtubesubtitles.yaml' 2026-05-11T15:53:43,577 adding 'lm_eval/tasks/pile_10k/README.md' 2026-05-11T15:53:43,578 adding 'lm_eval/tasks/pile_10k/pile_10k.yaml' 2026-05-11T15:53:43,580 adding 'lm_eval/tasks/piqa/README.md' 2026-05-11T15:53:43,581 adding 'lm_eval/tasks/piqa/piqa.yaml' 2026-05-11T15:53:43,583 adding 'lm_eval/tasks/pisa/README.md' 2026-05-11T15:53:43,584 adding 'lm_eval/tasks/pisa/_pisa.yaml' 2026-05-11T15:53:43,585 adding 'lm_eval/tasks/pisa/_pisa_llm_judged.yaml' 2026-05-11T15:53:43,587 adding 'lm_eval/tasks/pisa/_template_yaml' 2026-05-11T15:53:43,588 adding 'lm_eval/tasks/pisa/pisa_ch.yaml' 2026-05-11T15:53:43,589 adding 'lm_eval/tasks/pisa/pisa_ch_llm_judged.yaml' 2026-05-11T15:53:43,590 adding 'lm_eval/tasks/pisa/pisa_de.yaml' 2026-05-11T15:53:43,591 adding 'lm_eval/tasks/pisa/pisa_de_llm_judged.yaml' 2026-05-11T15:53:43,593 adding 'lm_eval/tasks/pisa/pisa_en.yaml' 2026-05-11T15:53:43,594 adding 'lm_eval/tasks/pisa/pisa_en_llm_judged.yaml' 2026-05-11T15:53:43,595 adding 'lm_eval/tasks/pisa/pisa_es.yaml' 2026-05-11T15:53:43,596 adding 'lm_eval/tasks/pisa/pisa_es_llm_judged.yaml' 2026-05-11T15:53:43,597 adding 'lm_eval/tasks/pisa/pisa_fr.yaml' 2026-05-11T15:53:43,599 adding 'lm_eval/tasks/pisa/pisa_fr_llm_judged.yaml' 2026-05-11T15:53:43,600 adding 'lm_eval/tasks/pisa/pisa_it.yaml' 2026-05-11T15:53:43,601 adding 'lm_eval/tasks/pisa/pisa_it_llm_judged.yaml' 2026-05-11T15:53:43,603 adding 'lm_eval/tasks/pisa/utils.py' 2026-05-11T15:53:43,605 adding 'lm_eval/tasks/polemo2/README.md' 2026-05-11T15:53:43,606 adding 'lm_eval/tasks/polemo2/polemo2_in.yaml' 2026-05-11T15:53:43,607 adding 'lm_eval/tasks/polemo2/polemo2_out.yaml' 2026-05-11T15:53:43,609 adding 'lm_eval/tasks/portuguese_bench/README.md' 2026-05-11T15:53:43,611 adding 'lm_eval/tasks/portuguese_bench/assin_entailment.yaml' 2026-05-11T15:53:43,612 adding 'lm_eval/tasks/portuguese_bench/assin_paraphrase.yaml' 2026-05-11T15:53:43,613 adding 'lm_eval/tasks/portuguese_bench/portuguese_bench.yaml' 2026-05-11T15:53:43,615 adding 'lm_eval/tasks/portuguese_bench/flores_pt/_flores_common_yaml' 2026-05-11T15:53:43,616 adding 'lm_eval/tasks/portuguese_bench/flores_pt/create_yamls_flores_pt.py' 2026-05-11T15:53:43,617 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_ca-pt.yaml' 2026-05-11T15:53:43,618 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_de-pt.yaml' 2026-05-11T15:53:43,620 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_en-pt.yaml' 2026-05-11T15:53:43,621 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_es-pt.yaml' 2026-05-11T15:53:43,622 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_eu-pt.yaml' 2026-05-11T15:53:43,623 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_fr-pt.yaml' 2026-05-11T15:53:43,624 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_gl-pt.yaml' 2026-05-11T15:53:43,625 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_it-pt.yaml' 2026-05-11T15:53:43,626 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-ca.yaml' 2026-05-11T15:53:43,628 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-de.yaml' 2026-05-11T15:53:43,629 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-en.yaml' 2026-05-11T15:53:43,630 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-es.yaml' 2026-05-11T15:53:43,631 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-eu.yaml' 2026-05-11T15:53:43,632 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-fr.yaml' 2026-05-11T15:53:43,633 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-gl.yaml' 2026-05-11T15:53:43,634 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt-it.yaml' 2026-05-11T15:53:43,636 adding 'lm_eval/tasks/portuguese_bench/flores_pt/flores_pt.yaml' 2026-05-11T15:53:43,638 adding 'lm_eval/tasks/prost/README.md' 2026-05-11T15:53:43,639 adding 'lm_eval/tasks/prost/corypaik_prost.yaml' 2026-05-11T15:53:43,641 adding 'lm_eval/tasks/pubmedqa/README.md' 2026-05-11T15:53:43,642 adding 'lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py' 2026-05-11T15:53:43,643 adding 'lm_eval/tasks/pubmedqa/pubmedqa.yaml' 2026-05-11T15:53:43,645 adding 'lm_eval/tasks/qa4mre/README.md' 2026-05-11T15:53:43,646 adding 'lm_eval/tasks/qa4mre/preprocess_qa4mre.py' 2026-05-11T15:53:43,647 adding 'lm_eval/tasks/qa4mre/qa4mre_2011.yaml' 2026-05-11T15:53:43,649 adding 'lm_eval/tasks/qa4mre/qa4mre_2012.yaml' 2026-05-11T15:53:43,650 adding 'lm_eval/tasks/qa4mre/qa4mre_2013.yaml' 2026-05-11T15:53:43,652 adding 'lm_eval/tasks/qasper/README.md' 2026-05-11T15:53:43,653 adding 'lm_eval/tasks/qasper/bool.yaml' 2026-05-11T15:53:43,654 adding 'lm_eval/tasks/qasper/freeform.yaml' 2026-05-11T15:53:43,655 adding 'lm_eval/tasks/qasper/metrics.py' 2026-05-11T15:53:43,657 adding 'lm_eval/tasks/qasper/utils.py' 2026-05-11T15:53:43,659 adding 'lm_eval/tasks/race/README.md' 2026-05-11T15:53:43,660 adding 'lm_eval/tasks/race/preprocess_race.py' 2026-05-11T15:53:43,661 adding 'lm_eval/tasks/race/race.yaml' 2026-05-11T15:53:43,663 adding 'lm_eval/tasks/realtoxicityprompts/metric.py' 2026-05-11T15:53:43,665 adding 'lm_eval/tasks/realtoxicityprompts/realtoxicityprompts.yaml' 2026-05-11T15:53:43,667 adding 'lm_eval/tasks/ruler/README.md' 2026-05-11T15:53:43,668 adding 'lm_eval/tasks/ruler/common_utils.py' 2026-05-11T15:53:43,670 adding 'lm_eval/tasks/ruler/cwe.yaml' 2026-05-11T15:53:43,671 adding 'lm_eval/tasks/ruler/cwe_utils.py' 2026-05-11T15:53:43,673 adding 'lm_eval/tasks/ruler/essays.py' 2026-05-11T15:53:43,674 adding 'lm_eval/tasks/ruler/fwe.yaml' 2026-05-11T15:53:43,676 adding 'lm_eval/tasks/ruler/fwe_utils.py' 2026-05-11T15:53:43,677 adding 'lm_eval/tasks/ruler/niah_multikey_1.yaml' 2026-05-11T15:53:43,678 adding 'lm_eval/tasks/ruler/niah_multikey_2.yaml' 2026-05-11T15:53:43,679 adding 'lm_eval/tasks/ruler/niah_multikey_3.yaml' 2026-05-11T15:53:43,681 adding 'lm_eval/tasks/ruler/niah_multiquery.yaml' 2026-05-11T15:53:43,682 adding 'lm_eval/tasks/ruler/niah_multivalue.yaml' 2026-05-11T15:53:43,683 adding 'lm_eval/tasks/ruler/niah_single_1.yaml' 2026-05-11T15:53:43,684 adding 'lm_eval/tasks/ruler/niah_single_2.yaml' 2026-05-11T15:53:43,685 adding 'lm_eval/tasks/ruler/niah_single_3.yaml' 2026-05-11T15:53:43,687 adding 'lm_eval/tasks/ruler/niah_utils.py' 2026-05-11T15:53:43,689 adding 'lm_eval/tasks/ruler/prepare_niah.py' 2026-05-11T15:53:43,690 adding 'lm_eval/tasks/ruler/qa_hotpot.yaml' 2026-05-11T15:53:43,691 adding 'lm_eval/tasks/ruler/qa_squad.yaml' 2026-05-11T15:53:43,693 adding 'lm_eval/tasks/ruler/qa_utils.py' 2026-05-11T15:53:43,694 adding 'lm_eval/tasks/ruler/ruler.yaml' 2026-05-11T15:53:43,695 adding 'lm_eval/tasks/ruler/vt.yaml' 2026-05-11T15:53:43,697 adding 'lm_eval/tasks/ruler/vt_utils.py' 2026-05-11T15:53:43,699 adding 'lm_eval/tasks/sciq/README.md' 2026-05-11T15:53:43,700 adding 'lm_eval/tasks/sciq/sciq.yaml' 2026-05-11T15:53:43,702 adding 'lm_eval/tasks/score/NON_GREEDY.md' 2026-05-11T15:53:43,704 adding 'lm_eval/tasks/score/README.md' 2026-05-11T15:53:43,705 adding 'lm_eval/tasks/score/non_greedy.sh' 2026-05-11T15:53:43,706 adding 'lm_eval/tasks/score/non_greedy_summarizer.py' 2026-05-11T15:53:43,708 adding 'lm_eval/tasks/score/score_robustness.yaml' 2026-05-11T15:53:43,709 adding 'lm_eval/tasks/score/utils.py' 2026-05-11T15:53:43,712 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_aqua_rat.yaml' 2026-05-11T15:53:43,713 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_logiqa_en.yaml' 2026-05-11T15:53:43,714 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lsat_rc.yaml' 2026-05-11T15:53:43,715 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_ar.yaml' 2026-05-11T15:53:43,717 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_lstat_lr.yaml' 2026-05-11T15:53:43,718 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_en.yaml' 2026-05-11T15:53:43,719 adding 'lm_eval/tasks/score/agi_eval/non_greedy_robustness_agieval_sat_math.yaml' 2026-05-11T15:53:43,720 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_aqua_rat.yaml' 2026-05-11T15:53:43,722 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_logiqa_en.yaml' 2026-05-11T15:53:43,723 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_ar.yaml' 2026-05-11T15:53:43,724 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_lr.yaml' 2026-05-11T15:53:43,726 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_lsat_rc.yaml' 2026-05-11T15:53:43,727 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_en.yaml' 2026-05-11T15:53:43,728 adding 'lm_eval/tasks/score/agi_eval/option_order_robustness_agieval_sat_math.yaml' 2026-05-11T15:53:43,729 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_aqua_rat.yaml' 2026-05-11T15:53:43,731 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_logiqa_en.yaml' 2026-05-11T15:53:43,732 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lsat_rc.yaml' 2026-05-11T15:53:43,733 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_ar.yaml' 2026-05-11T15:53:43,735 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_lstat_lr.yaml' 2026-05-11T15:53:43,736 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_en.yaml' 2026-05-11T15:53:43,737 adding 'lm_eval/tasks/score/agi_eval/prompt_robustness_agieval_sat_math.yaml' 2026-05-11T15:53:43,739 adding 'lm_eval/tasks/score/agi_eval/prompt_templates.json' 2026-05-11T15:53:43,740 adding 'lm_eval/tasks/score/agi_eval/score_non_greedy_robustness_agieval.yaml' 2026-05-11T15:53:43,741 adding 'lm_eval/tasks/score/agi_eval/score_option_order_robustness_agieval.yaml' 2026-05-11T15:53:43,743 adding 'lm_eval/tasks/score/agi_eval/score_prompt_robustness_agieval.yaml' 2026-05-11T15:53:43,744 adding 'lm_eval/tasks/score/agi_eval/score_robustness_agieval.yaml' 2026-05-11T15:53:43,745 adding 'lm_eval/tasks/score/agi_eval/utils_agieval.py' 2026-05-11T15:53:43,749 adding 'lm_eval/tasks/score/math/math_grader.py' 2026-05-11T15:53:43,750 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_algebra.yaml' 2026-05-11T15:53:43,751 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_counting_and_prob.yaml' 2026-05-11T15:53:43,752 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_geometry.yaml' 2026-05-11T15:53:43,754 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_intermediate_algebra.yaml' 2026-05-11T15:53:43,755 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_num_theory.yaml' 2026-05-11T15:53:43,756 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_prealgebra.yaml' 2026-05-11T15:53:43,757 adding 'lm_eval/tasks/score/math/non_greedy_robustness_math_precalc.yaml' 2026-05-11T15:53:43,758 adding 'lm_eval/tasks/score/math/prompt_robustness_math_algebra.yaml' 2026-05-11T15:53:43,760 adding 'lm_eval/tasks/score/math/prompt_robustness_math_counting_and_prob.yaml' 2026-05-11T15:53:43,761 adding 'lm_eval/tasks/score/math/prompt_robustness_math_geometry.yaml' 2026-05-11T15:53:43,762 adding 'lm_eval/tasks/score/math/prompt_robustness_math_intermediate_algebra.yaml' 2026-05-11T15:53:43,763 adding 'lm_eval/tasks/score/math/prompt_robustness_math_num_theory.yaml' 2026-05-11T15:53:43,764 adding 'lm_eval/tasks/score/math/prompt_robustness_math_prealgebra.yaml' 2026-05-11T15:53:43,766 adding 'lm_eval/tasks/score/math/prompt_robustness_math_precalc.yaml' 2026-05-11T15:53:43,767 adding 'lm_eval/tasks/score/math/prompt_templates.json' 2026-05-11T15:53:43,768 adding 'lm_eval/tasks/score/math/score_non_greedy_robustness_math.yaml' 2026-05-11T15:53:43,770 adding 'lm_eval/tasks/score/math/score_prompt_robustness_math.yaml' 2026-05-11T15:53:43,771 adding 'lm_eval/tasks/score/math/score_robustness_math.yaml' 2026-05-11T15:53:43,772 adding 'lm_eval/tasks/score/math/to_be_fixed_questions.json' 2026-05-11T15:53:43,774 adding 'lm_eval/tasks/score/math/utils_math.py' 2026-05-11T15:53:43,776 adding 'lm_eval/tasks/score/mmlu_pro/prompt_templates.json' 2026-05-11T15:53:43,777 adding 'lm_eval/tasks/score/mmlu_pro/score_non_greedy_robustness_mmlu_pro.yaml' 2026-05-11T15:53:43,779 adding 'lm_eval/tasks/score/mmlu_pro/score_option_order_robustness_mmlu_pro.yaml' 2026-05-11T15:53:43,780 adding 'lm_eval/tasks/score/mmlu_pro/score_prompt_robustness_mmlu_pro.yaml' 2026-05-11T15:53:43,781 adding 'lm_eval/tasks/score/mmlu_pro/utils_mmlu_pro.py' 2026-05-11T15:53:43,783 adding 'lm_eval/tasks/scrolls/README.md' 2026-05-11T15:53:43,784 adding 'lm_eval/tasks/scrolls/scrolls_contractnli.yaml' 2026-05-11T15:53:43,786 adding 'lm_eval/tasks/scrolls/scrolls_govreport.yaml' 2026-05-11T15:53:43,787 adding 'lm_eval/tasks/scrolls/scrolls_narrativeqa.yaml' 2026-05-11T15:53:43,788 adding 'lm_eval/tasks/scrolls/scrolls_qasper.yaml' 2026-05-11T15:53:43,793 adding 'lm_eval/tasks/scrolls/scrolls_qmsum.yaml' 2026-05-11T15:53:43,794 adding 'lm_eval/tasks/scrolls/scrolls_quality.yaml' 2026-05-11T15:53:43,795 adding 'lm_eval/tasks/scrolls/scrolls_summscreenfd.yaml' 2026-05-11T15:53:43,797 adding 'lm_eval/tasks/scrolls/task.py' 2026-05-11T15:53:43,799 adding 'lm_eval/tasks/simple_cooccurrence_bias/README.md' 2026-05-11T15:53:43,801 adding 'lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias.yaml' 2026-05-11T15:53:43,802 adding 'lm_eval/tasks/simple_cooccurrence_bias/simple_cooccurrence_bias_gen.yaml' 2026-05-11T15:53:43,803 adding 'lm_eval/tasks/simple_cooccurrence_bias/utils.py' 2026-05-11T15:53:43,805 adding 'lm_eval/tasks/siqa/README.md' 2026-05-11T15:53:43,806 adding 'lm_eval/tasks/siqa/siqa.yaml' 2026-05-11T15:53:43,808 adding 'lm_eval/tasks/slr_bench/README.md' 2026-05-11T15:53:43,810 adding 'lm_eval/tasks/slr_bench/lm_eval_slr_bench.py' 2026-05-11T15:53:43,811 adding 'lm_eval/tasks/slr_bench/slr_bench_all.yaml' 2026-05-11T15:53:43,812 adding 'lm_eval/tasks/slr_bench/slr_bench_basic.yaml' 2026-05-11T15:53:43,813 adding 'lm_eval/tasks/slr_bench/slr_bench_common_yaml' 2026-05-11T15:53:43,814 adding 'lm_eval/tasks/slr_bench/slr_bench_easy.yaml' 2026-05-11T15:53:43,815 adding 'lm_eval/tasks/slr_bench/slr_bench_group.yaml' 2026-05-11T15:53:43,816 adding 'lm_eval/tasks/slr_bench/slr_bench_hard.yaml' 2026-05-11T15:53:43,817 adding 'lm_eval/tasks/slr_bench/slr_bench_medium.yaml' 2026-05-11T15:53:43,820 adding 'lm_eval/tasks/spanish_bench/README.md' 2026-05-11T15:53:43,821 adding 'lm_eval/tasks/spanish_bench/cocoteros_es.yaml' 2026-05-11T15:53:43,822 adding 'lm_eval/tasks/spanish_bench/copa_es.yaml' 2026-05-11T15:53:43,823 adding 'lm_eval/tasks/spanish_bench/escola.yaml' 2026-05-11T15:53:43,825 adding 'lm_eval/tasks/spanish_bench/mgsm_direct_es_spanish_bench.yaml' 2026-05-11T15:53:43,826 adding 'lm_eval/tasks/spanish_bench/openbookqa_es.yaml' 2026-05-11T15:53:43,827 adding 'lm_eval/tasks/spanish_bench/paws_es_spanish_bench.yaml' 2026-05-11T15:53:43,828 adding 'lm_eval/tasks/spanish_bench/spanish_bench.yaml' 2026-05-11T15:53:43,830 adding 'lm_eval/tasks/spanish_bench/utils.py' 2026-05-11T15:53:43,831 adding 'lm_eval/tasks/spanish_bench/wnli_es.yaml' 2026-05-11T15:53:43,832 adding 'lm_eval/tasks/spanish_bench/xlsum_es.yaml' 2026-05-11T15:53:43,834 adding 'lm_eval/tasks/spanish_bench/xnli_es_spanish_bench.yaml' 2026-05-11T15:53:43,836 adding 'lm_eval/tasks/spanish_bench/flores_es/_flores_common_yaml' 2026-05-11T15:53:43,838 adding 'lm_eval/tasks/spanish_bench/flores_es/create_yamls_flores_es.py' 2026-05-11T15:53:43,839 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_ca-es.yaml' 2026-05-11T15:53:43,840 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_de-es.yaml' 2026-05-11T15:53:43,841 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_en-es.yaml' 2026-05-11T15:53:43,842 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-ca.yaml' 2026-05-11T15:53:43,844 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-de.yaml' 2026-05-11T15:53:43,845 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-en.yaml' 2026-05-11T15:53:43,846 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-eu.yaml' 2026-05-11T15:53:43,847 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-fr.yaml' 2026-05-11T15:53:43,849 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-gl.yaml' 2026-05-11T15:53:43,850 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-it.yaml' 2026-05-11T15:53:43,851 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es-pt.yaml' 2026-05-11T15:53:43,852 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_es.yaml' 2026-05-11T15:53:43,854 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_eu-es.yaml' 2026-05-11T15:53:43,855 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_fr-es.yaml' 2026-05-11T15:53:43,856 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_gl-es.yaml' 2026-05-11T15:53:43,857 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_it-es.yaml' 2026-05-11T15:53:43,858 adding 'lm_eval/tasks/spanish_bench/flores_es/flores_pt-es.yaml' 2026-05-11T15:53:43,860 adding 'lm_eval/tasks/spanish_bench/phrases_es/_phrases_es_common' 2026-05-11T15:53:43,861 adding 'lm_eval/tasks/spanish_bench/phrases_es/phrases_es-va.yaml' 2026-05-11T15:53:43,863 adding 'lm_eval/tasks/spanish_bench/phrases_es/phrases_va-es.yaml' 2026-05-11T15:53:43,864 adding 'lm_eval/tasks/squad_completion/README.md' 2026-05-11T15:53:43,866 adding 'lm_eval/tasks/squad_completion/squad_completion.yaml' 2026-05-11T15:53:43,867 adding 'lm_eval/tasks/squad_completion/task.py' 2026-05-11T15:53:43,869 adding 'lm_eval/tasks/squadv2/README.md' 2026-05-11T15:53:43,871 adding 'lm_eval/tasks/squadv2/squadv2.yaml' 2026-05-11T15:53:43,872 adding 'lm_eval/tasks/squadv2/task.py' 2026-05-11T15:53:43,874 adding 'lm_eval/tasks/storycloze/README.md' 2026-05-11T15:53:43,876 adding 'lm_eval/tasks/storycloze/storycloze_2016.yaml' 2026-05-11T15:53:43,877 adding 'lm_eval/tasks/storycloze/storycloze_2018.yaml' 2026-05-11T15:53:43,879 adding 'lm_eval/tasks/super_glue/README.md' 2026-05-11T15:53:43,881 adding 'lm_eval/tasks/super_glue/boolq/default.yaml' 2026-05-11T15:53:43,882 adding 'lm_eval/tasks/super_glue/boolq/seq2seq.yaml' 2026-05-11T15:53:43,883 adding 'lm_eval/tasks/super_glue/boolq/t5-prompt.yaml' 2026-05-11T15:53:43,885 adding 'lm_eval/tasks/super_glue/cb/aggregate.py' 2026-05-11T15:53:43,886 adding 'lm_eval/tasks/super_glue/cb/default.yaml' 2026-05-11T15:53:43,887 adding 'lm_eval/tasks/super_glue/cb/t5-prompt.yaml' 2026-05-11T15:53:43,889 adding 'lm_eval/tasks/super_glue/cb/t5_utils.py' 2026-05-11T15:53:43,891 adding 'lm_eval/tasks/super_glue/copa/default.yaml' 2026-05-11T15:53:43,892 adding 'lm_eval/tasks/super_glue/copa/t5-prompt.yaml' 2026-05-11T15:53:43,893 adding 'lm_eval/tasks/super_glue/copa/utils.py' 2026-05-11T15:53:43,895 adding 'lm_eval/tasks/super_glue/multirc/default.yaml' 2026-05-11T15:53:43,896 adding 'lm_eval/tasks/super_glue/multirc/t5-prompt.yaml' 2026-05-11T15:53:43,898 adding 'lm_eval/tasks/super_glue/multirc/t5_utils.py' 2026-05-11T15:53:43,900 adding 'lm_eval/tasks/super_glue/record/default.yaml' 2026-05-11T15:53:43,901 adding 'lm_eval/tasks/super_glue/record/t5-prompt.yaml' 2026-05-11T15:53:43,902 adding 'lm_eval/tasks/super_glue/record/t5_utils.py' 2026-05-11T15:53:43,903 adding 'lm_eval/tasks/super_glue/record/util.py' 2026-05-11T15:53:43,905 adding 'lm_eval/tasks/super_glue/rte/default.yaml' 2026-05-11T15:53:43,906 adding 'lm_eval/tasks/super_glue/rte/t5-prompt.yaml' 2026-05-11T15:53:43,908 adding 'lm_eval/tasks/super_glue/wic/default.yaml' 2026-05-11T15:53:43,910 adding 'lm_eval/tasks/super_glue/wic/t5-prompt.yaml' 2026-05-11T15:53:43,911 adding 'lm_eval/tasks/super_glue/wsc/default.yaml' 2026-05-11T15:53:43,913 adding 'lm_eval/tasks/super_glue/wsc/preprocess_wsc.py' 2026-05-11T15:53:43,914 adding 'lm_eval/tasks/super_glue/wsc/t5-prompt.yaml' 2026-05-11T15:53:43,915 adding 'lm_eval/tasks/super_glue/wsc/t5_utils.py' 2026-05-11T15:53:43,917 adding 'lm_eval/tasks/swag/README.md' 2026-05-11T15:53:43,919 adding 'lm_eval/tasks/swag/swag.yaml' 2026-05-11T15:53:43,921 adding 'lm_eval/tasks/swde/README.md' 2026-05-11T15:53:43,922 adding 'lm_eval/tasks/swde/swde.yaml' 2026-05-11T15:53:43,924 adding 'lm_eval/tasks/swde/task.py' 2026-05-11T15:53:43,926 adding 'lm_eval/tasks/tinyBenchmarks/README.md' 2026-05-11T15:53:43,927 adding 'lm_eval/tasks/tinyBenchmarks/agg_functions.py' 2026-05-11T15:53:43,928 adding 'lm_eval/tasks/tinyBenchmarks/tinyArc.yaml' 2026-05-11T15:53:43,930 adding 'lm_eval/tasks/tinyBenchmarks/tinyBenchmarks.yaml' 2026-05-11T15:53:43,931 adding 'lm_eval/tasks/tinyBenchmarks/tinyGSM8k.yaml' 2026-05-11T15:53:43,932 adding 'lm_eval/tasks/tinyBenchmarks/tinyHellaswag.yaml' 2026-05-11T15:53:43,934 adding 'lm_eval/tasks/tinyBenchmarks/tinyMMLU.yaml' 2026-05-11T15:53:43,935 adding 'lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc1.yaml' 2026-05-11T15:53:43,937 adding 'lm_eval/tasks/tinyBenchmarks/tinyTruthfulQA_mc2.yaml' 2026-05-11T15:53:43,938 adding 'lm_eval/tasks/tinyBenchmarks/tinyWinogrande.yaml' 2026-05-11T15:53:43,939 adding 'lm_eval/tasks/tinyBenchmarks/utils_hellaswag.py' 2026-05-11T15:53:43,941 adding 'lm_eval/tasks/tinyBenchmarks/utils_truthfulqa.py' 2026-05-11T15:53:43,942 adding 'lm_eval/tasks/tinyBenchmarks/utils_winogrande.py' 2026-05-11T15:53:43,944 adding 'lm_eval/tasks/tmlu/README.md' 2026-05-11T15:53:43,946 adding 'lm_eval/tasks/tmlu/subject.tsv' 2026-05-11T15:53:43,948 adding 'lm_eval/tasks/tmlu/default/_default_template_yaml' 2026-05-11T15:53:43,950 adding 'lm_eval/tasks/tmlu/default/_generate_configs.py' 2026-05-11T15:53:43,951 adding 'lm_eval/tasks/tmlu/default/_tmlu.yaml' 2026-05-11T15:53:43,953 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_biology.yaml' 2026-05-11T15:53:43,954 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_chemistry.yaml' 2026-05-11T15:53:43,955 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_chinese.yaml' 2026-05-11T15:53:43,957 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_civics.yaml' 2026-05-11T15:53:43,958 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_geography.yaml' 2026-05-11T15:53:43,959 adding 'lm_eval/tasks/tmlu/default/tmlu_AST_history.yaml' 2026-05-11T15:53:43,960 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_biology.yaml' 2026-05-11T15:53:43,961 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_chemistry.yaml' 2026-05-11T15:53:43,963 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_chinese.yaml' 2026-05-11T15:53:43,964 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_civics.yaml' 2026-05-11T15:53:43,965 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_earth_science.yaml' 2026-05-11T15:53:43,966 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_geography.yaml' 2026-05-11T15:53:43,967 adding 'lm_eval/tasks/tmlu/default/tmlu_CAP_history.yaml' 2026-05-11T15:53:43,969 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_biology.yaml' 2026-05-11T15:53:43,970 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_chemistry.yaml' 2026-05-11T15:53:43,971 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_chinese.yaml' 2026-05-11T15:53:43,973 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_civics.yaml' 2026-05-11T15:53:43,974 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_earth_science.yaml' 2026-05-11T15:53:43,975 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_geography.yaml' 2026-05-11T15:53:43,977 adding 'lm_eval/tasks/tmlu/default/tmlu_GSAT_history.yaml' 2026-05-11T15:53:43,978 adding 'lm_eval/tasks/tmlu/default/tmlu_accountant.yaml' 2026-05-11T15:53:43,979 adding 'lm_eval/tasks/tmlu/default/tmlu_basic_traditional_chinese_medicine.yaml' 2026-05-11T15:53:43,981 adding 'lm_eval/tasks/tmlu/default/tmlu_clinical_psychologist.yaml' 2026-05-11T15:53:43,982 adding 'lm_eval/tasks/tmlu/default/tmlu_clinical_traditional_chinese_medicine.yaml' 2026-05-11T15:53:43,983 adding 'lm_eval/tasks/tmlu/default/tmlu_driving_rule.yaml' 2026-05-11T15:53:43,984 adding 'lm_eval/tasks/tmlu/default/tmlu_lawyer_qualification.yaml' 2026-05-11T15:53:43,986 adding 'lm_eval/tasks/tmlu/default/tmlu_nutritionist.yaml' 2026-05-11T15:53:43,987 adding 'lm_eval/tasks/tmlu/default/tmlu_taiwan_tourist_resources.yaml' 2026-05-11T15:53:43,988 adding 'lm_eval/tasks/tmlu/default/tmlu_teacher_qualification.yaml' 2026-05-11T15:53:43,989 adding 'lm_eval/tasks/tmlu/default/tmlu_tour_guide.yaml' 2026-05-11T15:53:43,991 adding 'lm_eval/tasks/tmlu/default/tmlu_tour_leader.yaml' 2026-05-11T15:53:43,992 adding 'lm_eval/tasks/tmlu/default/utils.py' 2026-05-11T15:53:43,994 adding 'lm_eval/tasks/tmmluplus/README.md' 2026-05-11T15:53:43,995 adding 'lm_eval/tasks/tmmluplus/subject.tsv' 2026-05-11T15:53:43,999 adding 'lm_eval/tasks/tmmluplus/default/_generate_configs.py' 2026-05-11T15:53:44,000 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus.yaml' 2026-05-11T15:53:44,001 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_STEM.yaml' 2026-05-11T15:53:44,003 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_humanities.yaml' 2026-05-11T15:53:44,004 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_other.yaml' 2026-05-11T15:53:44,005 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_social_sciences.yaml' 2026-05-11T15:53:44,006 adding 'lm_eval/tasks/tmmluplus/default/_tmmluplus_template_yaml' 2026-05-11T15:53:44,008 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_accounting.yaml' 2026-05-11T15:53:44,009 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_administrative_law.yaml' 2026-05-11T15:53:44,011 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_advance_chemistry.yaml' 2026-05-11T15:53:44,012 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_agriculture.yaml' 2026-05-11T15:53:44,013 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_anti_money_laundering.yaml' 2026-05-11T15:53:44,014 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_auditing.yaml' 2026-05-11T15:53:44,016 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_basic_medical_science.yaml' 2026-05-11T15:53:44,017 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_business_management.yaml' 2026-05-11T15:53:44,018 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml' 2026-05-11T15:53:44,019 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml' 2026-05-11T15:53:44,021 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml' 2026-05-11T15:53:44,022 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml' 2026-05-11T15:53:44,023 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml' 2026-05-11T15:53:44,024 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml' 2026-05-11T15:53:44,025 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml' 2026-05-11T15:53:44,026 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml' 2026-05-11T15:53:44,027 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml' 2026-05-11T15:53:44,029 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml' 2026-05-11T15:53:44,030 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml' 2026-05-11T15:53:44,031 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml' 2026-05-11T15:53:44,032 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml' 2026-05-11T15:53:44,033 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml' 2026-05-11T15:53:44,035 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml' 2026-05-11T15:53:44,036 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml' 2026-05-11T15:53:44,037 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml' 2026-05-11T15:53:44,038 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml' 2026-05-11T15:53:44,039 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml' 2026-05-11T15:53:44,040 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml' 2026-05-11T15:53:44,042 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml' 2026-05-11T15:53:44,043 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml' 2026-05-11T15:53:44,044 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml' 2026-05-11T15:53:44,045 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml' 2026-05-11T15:53:44,046 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_linear_algebra.yaml' 2026-05-11T15:53:44,047 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_logic_reasoning.yaml' 2026-05-11T15:53:44,049 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_macroeconomics.yaml' 2026-05-11T15:53:44,050 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_management_accounting.yaml' 2026-05-11T15:53:44,051 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_marketing_management.yaml' 2026-05-11T15:53:44,052 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_mechanical.yaml' 2026-05-11T15:53:44,053 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_music.yaml' 2026-05-11T15:53:44,055 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_national_protection.yaml' 2026-05-11T15:53:44,056 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_nautical_science.yaml' 2026-05-11T15:53:44,057 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_occupational_therapy_for_psychological_disorders.yaml' 2026-05-11T15:53:44,058 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_official_document_management.yaml' 2026-05-11T15:53:44,060 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_optometry.yaml' 2026-05-11T15:53:44,061 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_organic_chemistry.yaml' 2026-05-11T15:53:44,062 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacology.yaml' 2026-05-11T15:53:44,063 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_pharmacy.yaml' 2026-05-11T15:53:44,064 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_physical_education.yaml' 2026-05-11T15:53:44,066 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_physics.yaml' 2026-05-11T15:53:44,067 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_politic_science.yaml' 2026-05-11T15:53:44,068 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_real_estate.yaml' 2026-05-11T15:53:44,069 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_secondary_physics.yaml' 2026-05-11T15:53:44,070 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_statistics_and_machine_learning.yaml' 2026-05-11T15:53:44,071 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_taiwanese_hokkien.yaml' 2026-05-11T15:53:44,072 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_taxation.yaml' 2026-05-11T15:53:44,073 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_technical.yaml' 2026-05-11T15:53:44,074 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_three_principles_of_people.yaml' 2026-05-11T15:53:44,076 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_trade.yaml' 2026-05-11T15:53:44,077 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_traditional_chinese_medicine_clinical_medicine.yaml' 2026-05-11T15:53:44,078 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_trust_practice.yaml' 2026-05-11T15:53:44,079 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_ttqav2.yaml' 2026-05-11T15:53:44,081 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_chinese_language.yaml' 2026-05-11T15:53:44,082 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_design.yaml' 2026-05-11T15:53:44,083 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_mathematics.yaml' 2026-05-11T15:53:44,084 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_tve_natural_sciences.yaml' 2026-05-11T15:53:44,085 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pathology.yaml' 2026-05-11T15:53:44,086 adding 'lm_eval/tasks/tmmluplus/default/tmmluplus_veterinary_pharmacology.yaml' 2026-05-11T15:53:44,088 adding 'lm_eval/tasks/tmmluplus/default/utils.py' 2026-05-11T15:53:44,090 adding 'lm_eval/tasks/toxigen/README.md' 2026-05-11T15:53:44,091 adding 'lm_eval/tasks/toxigen/toxigen.yaml' 2026-05-11T15:53:44,092 adding 'lm_eval/tasks/toxigen/utils.py' 2026-05-11T15:53:44,094 adding 'lm_eval/tasks/translation/README.md' 2026-05-11T15:53:44,095 adding 'lm_eval/tasks/translation/iwslt2017_ar-en.yaml' 2026-05-11T15:53:44,097 adding 'lm_eval/tasks/translation/iwslt2017_en-ar.yaml' 2026-05-11T15:53:44,098 adding 'lm_eval/tasks/translation/utils.py' 2026-05-11T15:53:44,099 adding 'lm_eval/tasks/translation/wmt14_en-fr.yaml' 2026-05-11T15:53:44,100 adding 'lm_eval/tasks/translation/wmt14_fr-en.yaml' 2026-05-11T15:53:44,102 adding 'lm_eval/tasks/translation/wmt16_de-en.yaml' 2026-05-11T15:53:44,103 adding 'lm_eval/tasks/translation/wmt16_en-de.yaml' 2026-05-11T15:53:44,104 adding 'lm_eval/tasks/translation/wmt16_en-ro.yaml' 2026-05-11T15:53:44,105 adding 'lm_eval/tasks/translation/wmt16_ro-en.yaml' 2026-05-11T15:53:44,106 adding 'lm_eval/tasks/translation/wmt_common_yaml' 2026-05-11T15:53:44,108 adding 'lm_eval/tasks/triviaqa/README.md' 2026-05-11T15:53:44,110 adding 'lm_eval/tasks/triviaqa/default.yaml' 2026-05-11T15:53:44,112 adding 'lm_eval/tasks/truthfulqa/README.md' 2026-05-11T15:53:44,113 adding 'lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml' 2026-05-11T15:53:44,114 adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml' 2026-05-11T15:53:44,115 adding 'lm_eval/tasks/truthfulqa/truthfulqa_mc2.yaml' 2026-05-11T15:53:44,117 adding 'lm_eval/tasks/truthfulqa/utils.py' 2026-05-11T15:53:44,119 adding 'lm_eval/tasks/truthfulqa-multi/README.md' 2026-05-11T15:53:44,121 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_ca.yaml' 2026-05-11T15:53:44,122 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_common' 2026-05-11T15:53:44,123 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_en.yaml' 2026-05-11T15:53:44,124 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_es.yaml' 2026-05-11T15:53:44,126 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_eu.yaml' 2026-05-11T15:53:44,127 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_gen_gl.yaml' 2026-05-11T15:53:44,128 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_ca.yaml' 2026-05-11T15:53:44,129 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_en.yaml' 2026-05-11T15:53:44,130 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_es.yaml' 2026-05-11T15:53:44,131 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_eu.yaml' 2026-05-11T15:53:44,132 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc1_gl.yaml' 2026-05-11T15:53:44,133 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_ca.yaml' 2026-05-11T15:53:44,134 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_en.yaml' 2026-05-11T15:53:44,135 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_es.yaml' 2026-05-11T15:53:44,137 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_eu.yaml' 2026-05-11T15:53:44,138 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc2_gl.yaml' 2026-05-11T15:53:44,139 adding 'lm_eval/tasks/truthfulqa-multi/truthfulqa-multi_mc_common' 2026-05-11T15:53:44,141 adding 'lm_eval/tasks/truthfulqa-multi/utils.py' 2026-05-11T15:53:44,143 adding 'lm_eval/tasks/turblimp/README.md' 2026-05-11T15:53:44,144 adding 'lm_eval/tasks/turblimp/_template_yaml' 2026-05-11T15:53:44,145 adding 'lm_eval/tasks/turblimp/anaphor_agreement.yaml' 2026-05-11T15:53:44,146 adding 'lm_eval/tasks/turblimp/argument_structure_ditransitive.yaml' 2026-05-11T15:53:44,147 adding 'lm_eval/tasks/turblimp/argument_structure_transitive.yaml' 2026-05-11T15:53:44,148 adding 'lm_eval/tasks/turblimp/binding.yaml' 2026-05-11T15:53:44,149 adding 'lm_eval/tasks/turblimp/determiners.yaml' 2026-05-11T15:53:44,151 adding 'lm_eval/tasks/turblimp/ellipsis.yaml' 2026-05-11T15:53:44,152 adding 'lm_eval/tasks/turblimp/irregular_forms.yaml' 2026-05-11T15:53:44,153 adding 'lm_eval/tasks/turblimp/island_effects.yaml' 2026-05-11T15:53:44,154 adding 'lm_eval/tasks/turblimp/nominalization.yaml' 2026-05-11T15:53:44,155 adding 'lm_eval/tasks/turblimp/npi_licensing.yaml' 2026-05-11T15:53:44,156 adding 'lm_eval/tasks/turblimp/passives.yaml' 2026-05-11T15:53:44,157 adding 'lm_eval/tasks/turblimp/quantifiers.yaml' 2026-05-11T15:53:44,158 adding 'lm_eval/tasks/turblimp/relative_clauses.yaml' 2026-05-11T15:53:44,159 adding 'lm_eval/tasks/turblimp/scrambling.yaml' 2026-05-11T15:53:44,161 adding 'lm_eval/tasks/turblimp/subject_agreement.yaml' 2026-05-11T15:53:44,162 adding 'lm_eval/tasks/turblimp/suspended_affixation.yaml' 2026-05-11T15:53:44,163 adding 'lm_eval/tasks/turblimp/turblimp_group.yaml' 2026-05-11T15:53:44,165 adding 'lm_eval/tasks/turkishmmlu/README.md' 2026-05-11T15:53:44,167 adding 'lm_eval/tasks/turkishmmlu/config/Biology.yaml' 2026-05-11T15:53:44,168 adding 'lm_eval/tasks/turkishmmlu/config/Chemistry.yaml' 2026-05-11T15:53:44,170 adding 'lm_eval/tasks/turkishmmlu/config/Geography.yaml' 2026-05-11T15:53:44,171 adding 'lm_eval/tasks/turkishmmlu/config/History.yaml' 2026-05-11T15:53:44,172 adding 'lm_eval/tasks/turkishmmlu/config/Mathematics.yaml' 2026-05-11T15:53:44,173 adding 'lm_eval/tasks/turkishmmlu/config/Philosophy.yaml' 2026-05-11T15:53:44,174 adding 'lm_eval/tasks/turkishmmlu/config/Physics.yaml' 2026-05-11T15:53:44,176 adding 'lm_eval/tasks/turkishmmlu/config/Religion_and_Ethics.yaml' 2026-05-11T15:53:44,177 adding 'lm_eval/tasks/turkishmmlu/config/Turkish_Language_and_Literature.yaml' 2026-05-11T15:53:44,178 adding 'lm_eval/tasks/turkishmmlu/config/_turkishmmlu_default_yaml' 2026-05-11T15:53:44,180 adding 'lm_eval/tasks/turkishmmlu/config_cot/Biology.yaml' 2026-05-11T15:53:44,183 adding 'lm_eval/tasks/turkishmmlu/config_cot/Chemistry.yaml' 2026-05-11T15:53:44,184 adding 'lm_eval/tasks/turkishmmlu/config_cot/Geography.yaml' 2026-05-11T15:53:44,186 adding 'lm_eval/tasks/turkishmmlu/config_cot/History.yaml' 2026-05-11T15:53:44,187 adding 'lm_eval/tasks/turkishmmlu/config_cot/Mathematics.yaml' 2026-05-11T15:53:44,189 adding 'lm_eval/tasks/turkishmmlu/config_cot/Philosophy.yaml' 2026-05-11T15:53:44,191 adding 'lm_eval/tasks/turkishmmlu/config_cot/Physics.yaml' 2026-05-11T15:53:44,193 adding 'lm_eval/tasks/turkishmmlu/config_cot/Religion_and_Ethics.yaml' 2026-05-11T15:53:44,194 adding 'lm_eval/tasks/turkishmmlu/config_cot/Turkish_Language_and_Literature.yaml' 2026-05-11T15:53:44,195 adding 'lm_eval/tasks/turkishmmlu/config_cot/_turkishmmlu_cot_default_yaml' 2026-05-11T15:53:44,197 adding 'lm_eval/tasks/ulqa/README.md' 2026-05-11T15:53:44,199 adding 'lm_eval/tasks/ulqa/celep1.yaml' 2026-05-11T15:53:44,200 adding 'lm_eval/tasks/ulqa/celep2.yaml' 2026-05-11T15:53:44,201 adding 'lm_eval/tasks/ulqa/lambada_uyghur.yaml' 2026-05-11T15:53:44,202 adding 'lm_eval/tasks/ulqa/uleval.yaml' 2026-05-11T15:53:44,203 adding 'lm_eval/tasks/ulqa/ulqa.yaml' 2026-05-11T15:53:44,205 adding 'lm_eval/tasks/ulqa/ulqa_.yaml' 2026-05-11T15:53:44,206 adding 'lm_eval/tasks/ulqa/ulut/nug.yaml' 2026-05-11T15:53:44,208 adding 'lm_eval/tasks/ulqa/ulut/ulut.yaml' 2026-05-11T15:53:44,209 adding 'lm_eval/tasks/ulqa/ulut/wag.yaml' 2026-05-11T15:53:44,210 adding 'lm_eval/tasks/ulqa/ulut/wsm.yaml' 2026-05-11T15:53:44,211 adding 'lm_eval/tasks/ulqa/ulut/wub.yaml' 2026-05-11T15:53:44,213 adding 'lm_eval/tasks/ulqa/ulut/wum.yaml' 2026-05-11T15:53:44,215 adding 'lm_eval/tasks/unitxt/20_newsgroups.yaml' 2026-05-11T15:53:44,216 adding 'lm_eval/tasks/unitxt/README.md' 2026-05-11T15:53:44,217 adding 'lm_eval/tasks/unitxt/ag_news.yaml' 2026-05-11T15:53:44,218 adding 'lm_eval/tasks/unitxt/argument_topic.yaml' 2026-05-11T15:53:44,219 adding 'lm_eval/tasks/unitxt/atis.yaml' 2026-05-11T15:53:44,220 adding 'lm_eval/tasks/unitxt/banking77.yaml' 2026-05-11T15:53:44,222 adding 'lm_eval/tasks/unitxt/claim_stance_topic.yaml' 2026-05-11T15:53:44,223 adding 'lm_eval/tasks/unitxt/cnn_dailymail.yaml' 2026-05-11T15:53:44,224 adding 'lm_eval/tasks/unitxt/coedit_gec.yaml' 2026-05-11T15:53:44,225 adding 'lm_eval/tasks/unitxt/dbpedia_14.yaml' 2026-05-11T15:53:44,227 adding 'lm_eval/tasks/unitxt/doc_vqa.yaml' 2026-05-11T15:53:44,228 adding 'lm_eval/tasks/unitxt/ethos_binary.yaml' 2026-05-11T15:53:44,229 adding 'lm_eval/tasks/unitxt/financial_tweets.yaml' 2026-05-11T15:53:44,230 adding 'lm_eval/tasks/unitxt/law_stack_exchange.yaml' 2026-05-11T15:53:44,231 adding 'lm_eval/tasks/unitxt/ledgar.yaml' 2026-05-11T15:53:44,233 adding 'lm_eval/tasks/unitxt/medical_abstracts.yaml' 2026-05-11T15:53:44,234 adding 'lm_eval/tasks/unitxt/stsb.yaml' 2026-05-11T15:53:44,235 adding 'lm_eval/tasks/unitxt/task.py' 2026-05-11T15:53:44,237 adding 'lm_eval/tasks/unitxt/unfair_tos.yaml' 2026-05-11T15:53:44,238 adding 'lm_eval/tasks/unitxt/unitxt' 2026-05-11T15:53:44,239 adding 'lm_eval/tasks/unitxt/unitxt_multimodal' 2026-05-11T15:53:44,240 adding 'lm_eval/tasks/unitxt/xsum.yaml' 2026-05-11T15:53:44,241 adding 'lm_eval/tasks/unitxt/yahoo_answers_topics.yaml' 2026-05-11T15:53:44,243 adding 'lm_eval/tasks/unscramble/README.md' 2026-05-11T15:53:44,245 adding 'lm_eval/tasks/unscramble/anagrams1.yaml' 2026-05-11T15:53:44,246 adding 'lm_eval/tasks/unscramble/anagrams2.yaml' 2026-05-11T15:53:44,247 adding 'lm_eval/tasks/unscramble/cycle_letters.yaml' 2026-05-11T15:53:44,248 adding 'lm_eval/tasks/unscramble/random_insertion.yaml' 2026-05-11T15:53:44,249 adding 'lm_eval/tasks/unscramble/reversed_words.yaml' 2026-05-11T15:53:44,251 adding 'lm_eval/tasks/webqs/README.md' 2026-05-11T15:53:44,252 adding 'lm_eval/tasks/webqs/utils.py' 2026-05-11T15:53:44,254 adding 'lm_eval/tasks/webqs/webqs.yaml' 2026-05-11T15:53:44,256 adding 'lm_eval/tasks/wikitext/README.md' 2026-05-11T15:53:44,257 adding 'lm_eval/tasks/wikitext/preprocess_wikitext.py' 2026-05-11T15:53:44,258 adding 'lm_eval/tasks/wikitext/wikitext.yaml' 2026-05-11T15:53:44,261 adding 'lm_eval/tasks/winogender/README.md' 2026-05-11T15:53:44,262 adding 'lm_eval/tasks/winogender/utils.py' 2026-05-11T15:53:44,263 adding 'lm_eval/tasks/winogender/winogender.yaml' 2026-05-11T15:53:44,264 adding 'lm_eval/tasks/winogender/winogender_female.yaml' 2026-05-11T15:53:44,266 adding 'lm_eval/tasks/winogender/winogender_gotcha.yaml' 2026-05-11T15:53:44,267 adding 'lm_eval/tasks/winogender/winogender_gotcha_female.yaml' 2026-05-11T15:53:44,268 adding 'lm_eval/tasks/winogender/winogender_gotcha_male.yaml' 2026-05-11T15:53:44,269 adding 'lm_eval/tasks/winogender/winogender_male.yaml' 2026-05-11T15:53:44,270 adding 'lm_eval/tasks/winogender/winogender_neutral.yaml' 2026-05-11T15:53:44,272 adding 'lm_eval/tasks/winogrande/README.md' 2026-05-11T15:53:44,273 adding 'lm_eval/tasks/winogrande/default.yaml' 2026-05-11T15:53:44,274 adding 'lm_eval/tasks/winogrande/preprocess_winogrande.py' 2026-05-11T15:53:44,276 adding 'lm_eval/tasks/wmdp/README.md' 2026-05-11T15:53:44,278 adding 'lm_eval/tasks/wmdp/_default_template_yaml' 2026-05-11T15:53:44,279 adding 'lm_eval/tasks/wmdp/_wmdp.yaml' 2026-05-11T15:53:44,280 adding 'lm_eval/tasks/wmdp/wmdp_bio.yaml' 2026-05-11T15:53:44,281 adding 'lm_eval/tasks/wmdp/wmdp_chem.yaml' 2026-05-11T15:53:44,282 adding 'lm_eval/tasks/wmdp/wmdp_cyber.yaml' 2026-05-11T15:53:44,284 adding 'lm_eval/tasks/wmt2016/README.md' 2026-05-11T15:53:44,285 adding 'lm_eval/tasks/wmt2016/metrics.py' 2026-05-11T15:53:44,287 adding 'lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml' 2026-05-11T15:53:44,288 adding 'lm_eval/tasks/wsc273/README.md' 2026-05-11T15:53:44,290 adding 'lm_eval/tasks/wsc273/default.yaml' 2026-05-11T15:53:44,291 adding 'lm_eval/tasks/wsc273/utils.py' 2026-05-11T15:53:44,293 adding 'lm_eval/tasks/xcopa/README.md' 2026-05-11T15:53:44,294 adding 'lm_eval/tasks/xcopa/_xcopa.yaml' 2026-05-11T15:53:44,295 adding 'lm_eval/tasks/xcopa/default_et.yaml' 2026-05-11T15:53:44,297 adding 'lm_eval/tasks/xcopa/default_ht.yaml' 2026-05-11T15:53:44,298 adding 'lm_eval/tasks/xcopa/default_id.yaml' 2026-05-11T15:53:44,299 adding 'lm_eval/tasks/xcopa/default_it.yaml' 2026-05-11T15:53:44,300 adding 'lm_eval/tasks/xcopa/default_qu.yaml' 2026-05-11T15:53:44,301 adding 'lm_eval/tasks/xcopa/default_sw.yaml' 2026-05-11T15:53:44,302 adding 'lm_eval/tasks/xcopa/default_ta.yaml' 2026-05-11T15:53:44,303 adding 'lm_eval/tasks/xcopa/default_th.yaml' 2026-05-11T15:53:44,304 adding 'lm_eval/tasks/xcopa/default_tr.yaml' 2026-05-11T15:53:44,306 adding 'lm_eval/tasks/xcopa/default_vi.yaml' 2026-05-11T15:53:44,307 adding 'lm_eval/tasks/xcopa/default_zh.yaml' 2026-05-11T15:53:44,308 adding 'lm_eval/tasks/xcopa/utils.py' 2026-05-11T15:53:44,310 adding 'lm_eval/tasks/xnli/README.md' 2026-05-11T15:53:44,311 adding 'lm_eval/tasks/xnli/_xnli.yaml' 2026-05-11T15:53:44,313 adding 'lm_eval/tasks/xnli/utils.py' 2026-05-11T15:53:44,314 adding 'lm_eval/tasks/xnli/xnli_ar.yaml' 2026-05-11T15:53:44,316 adding 'lm_eval/tasks/xnli/xnli_bg.yaml' 2026-05-11T15:53:44,317 adding 'lm_eval/tasks/xnli/xnli_common_yaml' 2026-05-11T15:53:44,318 adding 'lm_eval/tasks/xnli/xnli_de.yaml' 2026-05-11T15:53:44,319 adding 'lm_eval/tasks/xnli/xnli_el.yaml' 2026-05-11T15:53:44,320 adding 'lm_eval/tasks/xnli/xnli_en.yaml' 2026-05-11T15:53:44,322 adding 'lm_eval/tasks/xnli/xnli_es.yaml' 2026-05-11T15:53:44,323 adding 'lm_eval/tasks/xnli/xnli_fr.yaml' 2026-05-11T15:53:44,324 adding 'lm_eval/tasks/xnli/xnli_hi.yaml' 2026-05-11T15:53:44,325 adding 'lm_eval/tasks/xnli/xnli_ru.yaml' 2026-05-11T15:53:44,326 adding 'lm_eval/tasks/xnli/xnli_sw.yaml' 2026-05-11T15:53:44,327 adding 'lm_eval/tasks/xnli/xnli_th.yaml' 2026-05-11T15:53:44,329 adding 'lm_eval/tasks/xnli/xnli_tr.yaml' 2026-05-11T15:53:44,330 adding 'lm_eval/tasks/xnli/xnli_ur.yaml' 2026-05-11T15:53:44,331 adding 'lm_eval/tasks/xnli/xnli_vi.yaml' 2026-05-11T15:53:44,332 adding 'lm_eval/tasks/xnli/xnli_zh.yaml' 2026-05-11T15:53:44,334 adding 'lm_eval/tasks/xnli_eu/README.md' 2026-05-11T15:53:44,335 adding 'lm_eval/tasks/xnli_eu/xnli_common_yaml' 2026-05-11T15:53:44,337 adding 'lm_eval/tasks/xnli_eu/xnli_eu.yaml' 2026-05-11T15:53:44,338 adding 'lm_eval/tasks/xnli_eu/xnli_eu_mt.yaml' 2026-05-11T15:53:44,339 adding 'lm_eval/tasks/xnli_eu/xnli_eu_native.yaml' 2026-05-11T15:53:44,341 adding 'lm_eval/tasks/xquad/README.md' 2026-05-11T15:53:44,342 adding 'lm_eval/tasks/xquad/utils.py' 2026-05-11T15:53:44,343 adding 'lm_eval/tasks/xquad/xquad_ar.yaml' 2026-05-11T15:53:44,344 adding 'lm_eval/tasks/xquad/xquad_common_yaml' 2026-05-11T15:53:44,345 adding 'lm_eval/tasks/xquad/xquad_de.yaml' 2026-05-11T15:53:44,347 adding 'lm_eval/tasks/xquad/xquad_el.yaml' 2026-05-11T15:53:44,348 adding 'lm_eval/tasks/xquad/xquad_en.yaml' 2026-05-11T15:53:44,349 adding 'lm_eval/tasks/xquad/xquad_es.yaml' 2026-05-11T15:53:44,350 adding 'lm_eval/tasks/xquad/xquad_hi.yaml' 2026-05-11T15:53:44,352 adding 'lm_eval/tasks/xquad/xquad_ro.yaml' 2026-05-11T15:53:44,353 adding 'lm_eval/tasks/xquad/xquad_ru.yaml' 2026-05-11T15:53:44,354 adding 'lm_eval/tasks/xquad/xquad_th.yaml' 2026-05-11T15:53:44,355 adding 'lm_eval/tasks/xquad/xquad_tr.yaml' 2026-05-11T15:53:44,356 adding 'lm_eval/tasks/xquad/xquad_vi.yaml' 2026-05-11T15:53:44,357 adding 'lm_eval/tasks/xquad/xquad_zh.yaml' 2026-05-11T15:53:44,359 adding 'lm_eval/tasks/xstorycloze/README.md' 2026-05-11T15:53:44,360 adding 'lm_eval/tasks/xstorycloze/_xstorycloze.yaml' 2026-05-11T15:53:44,362 adding 'lm_eval/tasks/xstorycloze/default_ar.yaml' 2026-05-11T15:53:44,363 adding 'lm_eval/tasks/xstorycloze/default_en.yaml' 2026-05-11T15:53:44,364 adding 'lm_eval/tasks/xstorycloze/default_es.yaml' 2026-05-11T15:53:44,365 adding 'lm_eval/tasks/xstorycloze/default_eu.yaml' 2026-05-11T15:53:44,366 adding 'lm_eval/tasks/xstorycloze/default_hi.yaml' 2026-05-11T15:53:44,367 adding 'lm_eval/tasks/xstorycloze/default_id.yaml' 2026-05-11T15:53:44,368 adding 'lm_eval/tasks/xstorycloze/default_my.yaml' 2026-05-11T15:53:44,369 adding 'lm_eval/tasks/xstorycloze/default_ru.yaml' 2026-05-11T15:53:44,370 adding 'lm_eval/tasks/xstorycloze/default_sw.yaml' 2026-05-11T15:53:44,371 adding 'lm_eval/tasks/xstorycloze/default_te.yaml' 2026-05-11T15:53:44,372 adding 'lm_eval/tasks/xstorycloze/default_zh.yaml' 2026-05-11T15:53:44,374 adding 'lm_eval/tasks/xwinograd/README.md' 2026-05-11T15:53:44,376 adding 'lm_eval/tasks/xwinograd/_xwinograd.yaml' 2026-05-11T15:53:44,377 adding 'lm_eval/tasks/xwinograd/utils.py' 2026-05-11T15:53:44,378 adding 'lm_eval/tasks/xwinograd/xwinograd_common_yaml' 2026-05-11T15:53:44,379 adding 'lm_eval/tasks/xwinograd/xwinograd_en.yaml' 2026-05-11T15:53:44,380 adding 'lm_eval/tasks/xwinograd/xwinograd_fr.yaml' 2026-05-11T15:53:44,382 adding 'lm_eval/tasks/xwinograd/xwinograd_jp.yaml' 2026-05-11T15:53:44,383 adding 'lm_eval/tasks/xwinograd/xwinograd_pt.yaml' 2026-05-11T15:53:44,384 adding 'lm_eval/tasks/xwinograd/xwinograd_ru.yaml' 2026-05-11T15:53:44,385 adding 'lm_eval/tasks/xwinograd/xwinograd_zh.yaml' 2026-05-11T15:53:44,389 adding 'lm_eval/tasks/zhoblimp/BA_BEI_subj_drop.yaml' 2026-05-11T15:53:44,390 adding 'lm_eval/tasks/zhoblimp/BA_deletion.yaml' 2026-05-11T15:53:44,391 adding 'lm_eval/tasks/zhoblimp/BA_duplicate_argument.yaml' 2026-05-11T15:53:44,392 adding 'lm_eval/tasks/zhoblimp/BA_inversion.yaml' 2026-05-11T15:53:44,394 adding 'lm_eval/tasks/zhoblimp/BA_meiba.yaml' 2026-05-11T15:53:44,395 adding 'lm_eval/tasks/zhoblimp/BA_negation.yaml' 2026-05-11T15:53:44,396 adding 'lm_eval/tasks/zhoblimp/BA_no_progressive.yaml' 2026-05-11T15:53:44,397 adding 'lm_eval/tasks/zhoblimp/BA_no_stative_verb.yaml' 2026-05-11T15:53:44,398 adding 'lm_eval/tasks/zhoblimp/BA_suo_adverbial_a.yaml' 2026-05-11T15:53:44,400 adding 'lm_eval/tasks/zhoblimp/BA_suo_adverbial_b.yaml' 2026-05-11T15:53:44,401 adding 'lm_eval/tasks/zhoblimp/BA_verb_le_a.yaml' 2026-05-11T15:53:44,402 adding 'lm_eval/tasks/zhoblimp/BA_verb_le_b.yaml' 2026-05-11T15:53:44,403 adding 'lm_eval/tasks/zhoblimp/BEI_construction_a.yaml' 2026-05-11T15:53:44,404 adding 'lm_eval/tasks/zhoblimp/BEI_construction_b.yaml' 2026-05-11T15:53:44,405 adding 'lm_eval/tasks/zhoblimp/BEI_deletion.yaml' 2026-05-11T15:53:44,406 adding 'lm_eval/tasks/zhoblimp/BEI_preposition.yaml' 2026-05-11T15:53:44,408 adding 'lm_eval/tasks/zhoblimp/PN_numP_a.yaml' 2026-05-11T15:53:44,409 adding 'lm_eval/tasks/zhoblimp/PN_numP_b.yaml' 2026-05-11T15:53:44,410 adding 'lm_eval/tasks/zhoblimp/README.md' 2026-05-11T15:53:44,411 adding 'lm_eval/tasks/zhoblimp/_template_yaml' 2026-05-11T15:53:44,412 adding 'lm_eval/tasks/zhoblimp/adjective_transitive_dui.yaml' 2026-05-11T15:53:44,413 adding 'lm_eval/tasks/zhoblimp/agent_animacy_adv.yaml' 2026-05-11T15:53:44,414 adding 'lm_eval/tasks/zhoblimp/agent_animacy_passive.yaml' 2026-05-11T15:53:44,416 adding 'lm_eval/tasks/zhoblimp/agent_animacy_subj.yaml' 2026-05-11T15:53:44,417 adding 'lm_eval/tasks/zhoblimp/agent_causative.yaml' 2026-05-11T15:53:44,418 adding 'lm_eval/tasks/zhoblimp/agent_deletion.yaml' 2026-05-11T15:53:44,419 adding 'lm_eval/tasks/zhoblimp/anaphor_gender_agreement.yaml' 2026-05-11T15:53:44,420 adding 'lm_eval/tasks/zhoblimp/anaphor_number_agreement.yaml' 2026-05-11T15:53:44,421 adding 'lm_eval/tasks/zhoblimp/causative_shi_ba.yaml' 2026-05-11T15:53:44,422 adding 'lm_eval/tasks/zhoblimp/classifier_noun_agreement.yaml' 2026-05-11T15:53:44,424 adding 'lm_eval/tasks/zhoblimp/classifier_noun_agreement_no_gap.yaml' 2026-05-11T15:53:44,425 adding 'lm_eval/tasks/zhoblimp/classifier_noun_subj.yaml' 2026-05-11T15:53:44,426 adding 'lm_eval/tasks/zhoblimp/control_modal_vs_raising_modal.yaml' 2026-05-11T15:53:44,427 adding 'lm_eval/tasks/zhoblimp/ellipsis_adj.yaml' 2026-05-11T15:53:44,428 adding 'lm_eval/tasks/zhoblimp/ellipsis_double_object.yaml' 2026-05-11T15:53:44,429 adding 'lm_eval/tasks/zhoblimp/ellipsis_n_bar_class.yaml' 2026-05-11T15:53:44,431 adding 'lm_eval/tasks/zhoblimp/existential_there_subject_raising.yaml' 2026-05-11T15:53:44,432 adding 'lm_eval/tasks/zhoblimp/fci_renhe_dou.yaml' 2026-05-11T15:53:44,433 adding 'lm_eval/tasks/zhoblimp/fci_renhe_prepP.yaml' 2026-05-11T15:53:44,434 adding 'lm_eval/tasks/zhoblimp/fci_renhe_ruguo.yaml' 2026-05-11T15:53:44,435 adding 'lm_eval/tasks/zhoblimp/fci_renhe_subj.yaml' 2026-05-11T15:53:44,437 adding 'lm_eval/tasks/zhoblimp/fci_renhe_suoyou.yaml' 2026-05-11T15:53:44,438 adding 'lm_eval/tasks/zhoblimp/intransitive_double_obj.yaml' 2026-05-11T15:53:44,439 adding 'lm_eval/tasks/zhoblimp/intransitive_no_obj.yaml' 2026-05-11T15:53:44,440 adding 'lm_eval/tasks/zhoblimp/left_adverbial_b.yaml' 2026-05-11T15:53:44,441 adding 'lm_eval/tasks/zhoblimp/left_adverbial_d.yaml' 2026-05-11T15:53:44,442 adding 'lm_eval/tasks/zhoblimp/left_adverbial_e.yaml' 2026-05-11T15:53:44,444 adding 'lm_eval/tasks/zhoblimp/left_adverbial_negation.yaml' 2026-05-11T15:53:44,445 adding 'lm_eval/tasks/zhoblimp/left_dou.yaml' 2026-05-11T15:53:44,446 adding 'lm_eval/tasks/zhoblimp/modal_raising_hui.yaml' 2026-05-11T15:53:44,447 adding 'lm_eval/tasks/zhoblimp/modal_raising_topicalization.yaml' 2026-05-11T15:53:44,449 adding 'lm_eval/tasks/zhoblimp/nominal_definite_men.yaml' 2026-05-11T15:53:44,450 adding 'lm_eval/tasks/zhoblimp/nominal_modal_insertion.yaml' 2026-05-11T15:53:44,451 adding 'lm_eval/tasks/zhoblimp/noun_adjective_shi.yaml' 2026-05-11T15:53:44,452 adding 'lm_eval/tasks/zhoblimp/noun_phrase_conjunction_jian.yaml' 2026-05-11T15:53:44,453 adding 'lm_eval/tasks/zhoblimp/npi_renhe_A_not_A_question.yaml' 2026-05-11T15:53:44,455 adding 'lm_eval/tasks/zhoblimp/npi_renhe_conditional.yaml' 2026-05-11T15:53:44,456 adding 'lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_locP.yaml' 2026-05-11T15:53:44,457 adding 'lm_eval/tasks/zhoblimp/npi_renhe_neg_scope_subj.yaml' 2026-05-11T15:53:44,458 adding 'lm_eval/tasks/zhoblimp/npi_renhe_wh_question_obj.yaml' 2026-05-11T15:53:44,460 adding 'lm_eval/tasks/zhoblimp/npi_renhe_wh_question_subj.yaml' 2026-05-11T15:53:44,461 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_long_left.yaml' 2026-05-11T15:53:44,462 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_a.yaml' 2026-05-11T15:53:44,463 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_long_right_b.yaml' 2026-05-11T15:53:44,464 adding 'lm_eval/tasks/zhoblimp/passive_agent_deletion_short.yaml' 2026-05-11T15:53:44,466 adding 'lm_eval/tasks/zhoblimp/passive_body_part.yaml' 2026-05-11T15:53:44,467 adding 'lm_eval/tasks/zhoblimp/passive_intransitive.yaml' 2026-05-11T15:53:44,468 adding 'lm_eval/tasks/zhoblimp/passive_no_adj.yaml' 2026-05-11T15:53:44,469 adding 'lm_eval/tasks/zhoblimp/passive_suo.yaml' 2026-05-11T15:53:44,470 adding 'lm_eval/tasks/zhoblimp/plural_cardinal_men_a.yaml' 2026-05-11T15:53:44,471 adding 'lm_eval/tasks/zhoblimp/plural_cardinal_men_b.yaml' 2026-05-11T15:53:44,472 adding 'lm_eval/tasks/zhoblimp/preposition_deletion.yaml' 2026-05-11T15:53:44,473 adding 'lm_eval/tasks/zhoblimp/preposition_insertion.yaml' 2026-05-11T15:53:44,475 adding 'lm_eval/tasks/zhoblimp/principle_A_c_command.yaml' 2026-05-11T15:53:44,476 adding 'lm_eval/tasks/zhoblimp/principle_A_c_command_number.yaml' 2026-05-11T15:53:44,477 adding 'lm_eval/tasks/zhoblimp/principle_A_domain.yaml' 2026-05-11T15:53:44,478 adding 'lm_eval/tasks/zhoblimp/principle_A_domain_number.yaml' 2026-05-11T15:53:44,479 adding 'lm_eval/tasks/zhoblimp/question_A_not_A.yaml' 2026-05-11T15:53:44,481 adding 'lm_eval/tasks/zhoblimp/question_A_not_A_daodi_a.yaml' 2026-05-11T15:53:44,482 adding 'lm_eval/tasks/zhoblimp/question_A_not_A_daodi_b.yaml' 2026-05-11T15:53:44,483 adding 'lm_eval/tasks/zhoblimp/question_A_not_A_indirect.yaml' 2026-05-11T15:53:44,484 adding 'lm_eval/tasks/zhoblimp/question_V_not_VP_1.yaml' 2026-05-11T15:53:44,485 adding 'lm_eval/tasks/zhoblimp/question_V_not_VP_2.yaml' 2026-05-11T15:53:44,486 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_1.yaml' 2026-05-11T15:53:44,487 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_2.yaml' 2026-05-11T15:53:44,488 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_intran.yaml' 2026-05-11T15:53:44,489 adding 'lm_eval/tasks/zhoblimp/question_daodi_nandao_A_not_A_tran.yaml' 2026-05-11T15:53:44,491 adding 'lm_eval/tasks/zhoblimp/question_daodi_negation.yaml' 2026-05-11T15:53:44,492 adding 'lm_eval/tasks/zhoblimp/question_nandao_negation.yaml' 2026-05-11T15:53:44,493 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_1_a.yaml' 2026-05-11T15:53:44,494 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_1_b.yaml' 2026-05-11T15:53:44,495 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_2.yaml' 2026-05-11T15:53:44,497 adding 'lm_eval/tasks/zhoblimp/question_nandao_raising_3.yaml' 2026-05-11T15:53:44,498 adding 'lm_eval/tasks/zhoblimp/question_nandao_scope_1.yaml' 2026-05-11T15:53:44,499 adding 'lm_eval/tasks/zhoblimp/question_nandao_scope_2.yaml' 2026-05-11T15:53:44,500 adding 'lm_eval/tasks/zhoblimp/question_particle_daodi_choice_intran.yaml' 2026-05-11T15:53:44,501 adding 'lm_eval/tasks/zhoblimp/question_particle_daodi_choice_tran.yaml' 2026-05-11T15:53:44,503 adding 'lm_eval/tasks/zhoblimp/question_particle_nandao.yaml' 2026-05-11T15:53:44,504 adding 'lm_eval/tasks/zhoblimp/relative_operator_intepretation.yaml' 2026-05-11T15:53:44,505 adding 'lm_eval/tasks/zhoblimp/relative_operator_who.yaml' 2026-05-11T15:53:44,506 adding 'lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml' 2026-05-11T15:53:44,507 adding 'lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml' 2026-05-11T15:53:44,509 adding 'lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml' 2026-05-11T15:53:44,510 adding 'lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml' 2026-05-11T15:53:44,511 adding 'lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml' 2026-05-11T15:53:44,512 adding 'lm_eval/tasks/zhoblimp/right_yijing_a.yaml' 2026-05-11T15:53:44,513 adding 'lm_eval/tasks/zhoblimp/right_yijing_b.yaml' 2026-05-11T15:53:44,514 adding 'lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml' 2026-05-11T15:53:44,516 adding 'lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml' 2026-05-11T15:53:44,517 adding 'lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml' 2026-05-11T15:53:44,518 adding 'lm_eval/tasks/zhoblimp/topicalization_OSV.yaml' 2026-05-11T15:53:44,519 adding 'lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml' 2026-05-11T15:53:44,520 adding 'lm_eval/tasks/zhoblimp/topicalization_SOV.yaml' 2026-05-11T15:53:44,521 adding 'lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml' 2026-05-11T15:53:44,522 adding 'lm_eval/tasks/zhoblimp/verb_negation_particle.yaml' 2026-05-11T15:53:44,523 adding 'lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml' 2026-05-11T15:53:44,524 adding 'lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml' 2026-05-11T15:53:44,525 adding 'lm_eval/tasks/zhoblimp/ya_insertion.yaml' 2026-05-11T15:53:44,526 adding 'lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml' 2026-05-11T15:53:44,528 adding 'lm_eval/tasks/zhoblimp/you_yige.yaml' 2026-05-11T15:53:44,529 adding 'lm_eval/tasks/zhoblimp/zhoblimp_group.yaml' 2026-05-11T15:53:44,532 adding 'lm_eval-0.5.0.dev1.dist-info/licenses/LICENSE.md' 2026-05-11T15:53:44,539 adding 'lm_eval-0.5.0.dev1.dist-info/METADATA' 2026-05-11T15:53:44,540 adding 'lm_eval-0.5.0.dev1.dist-info/WHEEL' 2026-05-11T15:53:44,541 adding 'lm_eval-0.5.0.dev1.dist-info/entry_points.txt' 2026-05-11T15:53:44,542 adding 'lm_eval-0.5.0.dev1.dist-info/top_level.txt' 2026-05-11T15:53:44,799 adding 'lm_eval-0.5.0.dev1.dist-info/RECORD' 2026-05-11T15:53:45,049 removing build/bdist.linux-armv7l/wheel 2026-05-11T15:53:49,187 Building wheel for lm-eval (pyproject.toml): finished with status 'done' 2026-05-11T15:53:49,359 Created wheel for lm-eval: filename=lm_eval-0.5.0.dev1-py3-none-any.whl size=8824039 sha256=2b123742db0404ed529819f64cdd873004c66e5d8ad3d552a5e5c5f7f216d246 2026-05-11T15:53:49,360 Stored in directory: /tmp/pip-ephem-wheel-cache-drshsi6r/wheels/ec/5b/e0/c3390b2a5023cdd90e7a21ee8cf20e09c4115b3fee5f4dc644 2026-05-11T15:53:49,830 Successfully built lm-eval 2026-05-11T15:53:50,014 Removed build tracker: '/tmp/pip-build-tracker-gray8eve'